# Autoencoder

둘러보기로 가기 검색하러 가기

## 노트

### 말뭉치

1. An autoencoder is a special type of neural network that is trained to copy its input to its output.[1]
2. For example, given an image of a handwritten digit, an autoencoder first encodes the image into a lower dimensional latent representation, then decodes the latent representation back to an image.[1]
3. To start, you will train the basic autoencoder using the Fashon MNIST dataset.[1]
4. An autoencoder can also be trained to remove noise from images.[1]
5. Autoencoders are an unsupervised learning technique in which we leverage neural networks for the task of representation learning.[2]
6. An undercomplete autoencoder has no explicit regularization term - we simply train our model according to the reconstruction loss.[2]
7. For deep autoencoders, we must also be aware of the capacity of our encoder and decoder models.[2]
8. Sparse autoencoders offer us an alternative method for introducing an information bottleneck without requiring a reduction in the number of nodes at our hidden layers.[2]
9. An autoencoder neural network is an unsupervised learning algorithm that applies backpropagation, setting the target values to be equal to the inputs.[3]
10. Recall that \textstyle a^{(2)}_j denotes the activation of hidden unit \textstyle j in the autoencoder.[3]
11. Having trained a (sparse) autoencoder, we would now like to visualize the function learned by the algorithm, to try to understand what it has learned.[3]
12. Consider the case of training an autoencoder on \textstyle 10 \times 10 images, so that \textstyle n = 100 .[3]
13. 오토인코더에 대해 개념과 uncomplete, stacked, denoising, sparse, VAE 오토인코더에 대해 알아보았다.[4]
14. This article will cover the most common use cases for Autoencoder.[5]
15. The network architecture for autoencoders can vary between a simple FeedForward network, LSTM network or Convolutional Neural Network depending on the use case.[5]
16. Let’s say that we have trained an autoencoder on the MNIST dataset.[5]
17. The code below uses two different images to predict the anomaly score (reconstruction error) using the autoencoder network we trained above.[5]
18. An autoencoder consists of 3 components: encoder, code and decoder.[6]
19. To build an autoencoder we need 3 things: an encoding method, decoding method, and a loss function to compare the output with the target.[6]
20. Lossy: The output of the autoencoder will not be exactly the same as the input, it will be a close but degraded representation.[6]
21. Unsupervised: To train an autoencoder we don’t need to do anything fancy, just throw the raw input data at it.[6]
22. The aim of an autoencoder is to learn a representation (encoding) for a set of data, typically for dimensionality reduction, by training the network to ignore signal “noise”.[7]
23. An autoencoder is a neural network that learns to copy its input to its output.[7]
24. The k-sparse autoencoder is based on a linear autoencoder (i.e. with linear activation function) and tied weights.[7]
25. In practice, the objective of denoising autoencoders is that of cleaning the corrupted input, or denoising.[7]
26. ) Autoencoders are data-specific, which means that they will only be able to compress data similar to what they have been trained on.[8]
27. Autoencoders are lossy, which means that the decompressed outputs will be degraded compared to the original inputs (similar to MP3 or JPEG compression).[8]
28. Today two interesting practical applications of autoencoders are data denoising (which we feature later in this post), and dimensionality reduction for data visualization.[8]
29. As a result, a lot of newcomers to the field absolutely love autoencoders and can't get enough of them.[8]
30. A variational autoencoder (VAE) provides a probabilistic manner for describing an observation in latent space, such that the encoder describes a probability distribution for each latent attribute.[9]
31. With this bottleneck structure an autoencoder learns to extract the most important information when the input goes through the latent layers.[9]
32. Therefore, an autoencoder is an effective way to project data from a high dimension to a lower dimension by extracting the most dominant features and characteristics.[9]
33. As mentioned earlier, the synthesis of NMR T2 distributions using VAE-NN requires a two-stage training process prior to the testing and deploying the neural network (Fig. 7.2).[9]
34. For more information on autoencoders and other neural network based approaches, see the work of Geoffrey Hinton (Hinton & Salakhutdinov, 2006)3 and many others.[10]
35. The autoencoder network has three layers: the input, a hidden layer for encoding, and the output decoding layer.[11]
36. Autoencoder networks teach themselves how to compress data from the input layer into a shorter code, and then uncompress that code into whatever format best matches the original input.[11]
37. Denoising autoencoder - Using a partially corrupted input to learn how to recover the original undistorted input.[11]
38. If you’ve read about unsupervised learning techniques before, you may have come across the term “autoencoder”.[12]
39. Autoencoders are one of the primary ways that unsupervised learning models are developed.[12]
40. Briefly, autoencoders operate by taking in data, compressing and encoding the data, and then reconstructing the data from the encoding representation.[12]
41. Through this process, an autoencoder can learn the important features of the data.[12]
42. In addition, the autoencoder is explicitly optimized for the data reconstruction from the code.[13]
43. To avoid overfitting and improve the robustness, Denoising Autoencoder (Vincent et al. 2008) proposed a modification to the basic autoencoder.[13]
44. Sparse Autoencoder applies a “sparse” constraint on the hidden unit activation to avoid overfitting and improve robustness.[13]
45. In $$k$$-Sparse Autoencoder (Makhzani and Frey, 2013), the sparsity is enforced by only keeping the top k highest activations in the bottleneck layer with linear activation function.[13]
46. To analyze such data, several machine learning, bioinformatics, and statistical methods have been applied, among them neural networks such as autoencoders.[14]
47. In this paper, we investigate several autoencoder architectures that integrate a variety of cancer patient data types (e.g., multi-omics and clinical data).[14]
48. In this paper we design and systematically analyze several deep-learning approaches for data integration based on Variational Autoencoders (VAEs) (Kingma and Welling, 2014).[14]
49. Autoencoders learn a compressed representation (embedding/code) of the input data by reconstructing it on the output of the network.[14]
50. The best performing model was the Composite Model that combined an autoencoder and a future predictor.[15]
51. 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 # lstm autoencoder recreate sequence from numpy import array from keras .[15]
52. 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 # lstm autoencoder predict sequence from numpy import array from keras .[15]
53. 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 # lstm autoencoder reconstruct and predict sequence from numpy import array from keras .[15]
54. Processing the benchmark dataset MNIST, a deep autoencoder would use binary transformations after each RBM.[16]
55. The decoding half of a deep autoencoder is a feed-forward net with layers 100, 250, 500 and 1000 nodes wide, respectively.[16]
56. The decoding half of a deep autoencoder is the part that learns to reconstruct the image.[16]
57. The scaled word counts are then fed into a deep-belief network, a stack of restricted Boltzmann machines, which themselves are just a subset of feedforward-backprop autoencoders.[16]
58. Generally, you can consider autoencoders as an unsupervised learning technique, since you don’t need explicit labels to train the model on.[17]
59. In this tutorial, you’ll learn about autoencoders in deep learning and you will implement a convolutional and denoising autoencoder in Python with Keras.[17]
60. The compression in autoencoders is achieved by training the network for a period of time and as it learns it tries to best represent the input image at the bottleneck.[17]
61. You feed an image with just five pixel values into the autoencoder which is compressed by the encoder into three pixel values at the bottleneck (middle layer) or latent space.[17]
62. Autoencoder is a wildly used deep learning architecture.[18]
63. Recent studies focused on modifying the autoencoder algorithm to solve the two challenges.[18]
64. The estimation of the model was done by expectation maximization (EM), but it should be easy for autoencoder to do the job, as pointed out that EM=VAE.[18]
65. Autoencoder can also be used for supervised learning, similar to principal component regression (PCR).[18]
66. After the autoencoder was trained on the training set, we obtained the superset outputs for the training and test sets.[19]
67. In the first part of this tutorial, we’ll discuss what autoencoders are, including how convolutional autoencoders can be applied to image data.[20]
68. If the goal of an autoencoder is just to reconstruct the input, why even use the network in the first place?[20]
69. Later in this tutorial, we’ll be training an autoencoder on the MNIST dataset.[20]
70. Autoencoders cannot generate new, realistic data points that could be considered “passable” by humans.[20]
71. An autoencoder is a type of artificial neural network used to learn efficient data codings in an unsupervised manner.[21]
72. Along with the reduction side, a reconstructing side is also learned, where the autoencoder tries to generate from the reduced encoding a representation as close as possible to its original input.[21]
73. Denoising autoencoders create a corrupted copy of the input by introducing some noise.[21]
74. This helps to avoid the autoencoders to copy the input to the output without learning features about the data.[21]
75. The methodological hierarchy in this work was based on the autoencoder framework, and 12 sampling sizes were considered for landslide susceptibility mapping.[22]
76. The final prediction results obtained from the autoencoder modeling were evaluated using the testing data set based on qualitative and quantitative analyses to validate the performance of the models.[22]
77. A flowchart of the proposed autoencoder framework is illustrated in Fig.[22]
78. The autoencoder is trained to reconstruct input of landslide influencing factors onto the output layer for feature representation, which prevents the simple copying of the data and the network.[22]
79. For those getting started with neural networks, autoencoders can look and sound intimidating.[23]
80. First and foremost, autoencoders are trained via unsupervised learning, which means you don't need labels.[23]
81. An autoencoder is learned to predict its own input by using a noisy version of itself, which forces it to take advantage of structure in the data to learn compact ways of representing it.[23]
82. To the best of our knowledge, this research is the first to implement stacked autoencoders by using DAEs and AEs for feature learning in DL.[24]
83. Autoencoders are unsupervised neural networks that use machine learning to do this compression for us.[25]
84. An autoencoder neural network is an Unsupervised Machine learning algorithm that applies backpropagation, setting the target values to be equal to the inputs.[25]
85. Autoencoders are used to reduce the size of our inputs into a smaller representation.[25]
86. So you might be thinking why do we need Autoencoders then?[25]
87. Configure the VAE to use the specified loss function for the reconstruction, instead of a ReconstructionDistribution.[26]
88. Note that this is NOT following the standard VAE design (as per Kingma & Welling), which assumes a probabilistic output - i.e., some p(x|z).[26]
89. Set the number of samples per data point (from VAE state Z) used when doing pretraining.[26]
90. In this blog we’ve talked about autoencoders several times, both as outliers detection and as dimensionality reduction.[27]
91. Now, we present another variation of them, variational autoencoder, which makes possible data augmentation.[27]
92. As a kind reminder, an autoencoder network is composed of a pair of two connected networks: an encoder and a decoder.[27]
93. But the hidden layer in autoencoders may not be continuous, which might make difficult interpolation.[27]
94. The generator takes the form of a fully convolutional autoencoder.[28]
95. The autoencoder (left side of diagram) accepts a masked image as an input, and attempts to reconstruct the original unmasked image.[28]
96. The discriminator is run using the output of the autoencoder.[28]
97. The result is used to influence the cost function used to update the autoencoder's weights.[28]
98. We then use autoencoders to reduce the spectra feature dimensions from 1851 to 10 and re-train the ANN models.[29]
99. An autoencoder and artificial neural network-based method to estimate parity status of wild mosquitoes from near-infrared spectra.[29]
100. We then apply autoencoders to reduce the spectra feature space from 1851 to 10 and re-train ANN models.[29]
101. The ANN model achieved an average accuracy of 72% and 93% before and after applying the autoencoder, respectively.[29]