# Autoencoder

둘러보기로 가기
검색하러 가기

## 노트

### 위키데이터

- ID : Q786435

### 말뭉치

- An autoencoder is a special type of neural network that is trained to copy its input to its output.
^{[1]} - For example, given an image of a handwritten digit, an autoencoder first encodes the image into a lower dimensional latent representation, then decodes the latent representation back to an image.
^{[1]} - To start, you will train the basic autoencoder using the Fashon MNIST dataset.
^{[1]} - An autoencoder can also be trained to remove noise from images.
^{[1]} - Autoencoders are an unsupervised learning technique in which we leverage neural networks for the task of representation learning.
^{[2]} - An undercomplete autoencoder has no explicit regularization term - we simply train our model according to the reconstruction loss.
^{[2]} - For deep autoencoders, we must also be aware of the capacity of our encoder and decoder models.
^{[2]} - Sparse autoencoders offer us an alternative method for introducing an information bottleneck without requiring a reduction in the number of nodes at our hidden layers.
^{[2]} - An autoencoder neural network is an unsupervised learning algorithm that applies backpropagation, setting the target values to be equal to the inputs.
^{[3]} - Recall that \textstyle a^{(2)}_j denotes the activation of hidden unit \textstyle j in the autoencoder.
^{[3]} - Having trained a (sparse) autoencoder, we would now like to visualize the function learned by the algorithm, to try to understand what it has learned.
^{[3]} - Consider the case of training an autoencoder on \textstyle 10 \times 10 images, so that \textstyle n = 100 .
^{[3]} - 오토인코더에 대해 개념과 uncomplete, stacked, denoising, sparse, VAE 오토인코더에 대해 알아보았다.
^{[4]} - This article will cover the most common use cases for Autoencoder.
^{[5]} - The network architecture for autoencoders can vary between a simple FeedForward network, LSTM network or Convolutional Neural Network depending on the use case.
^{[5]} - Let’s say that we have trained an autoencoder on the MNIST dataset.
^{[5]} - The code below uses two different images to predict the anomaly score (reconstruction error) using the autoencoder network we trained above.
^{[5]} - An autoencoder consists of 3 components: encoder, code and decoder.
^{[6]} - To build an autoencoder we need 3 things: an encoding method, decoding method, and a loss function to compare the output with the target.
^{[6]} - Lossy: The output of the autoencoder will not be exactly the same as the input, it will be a close but degraded representation.
^{[6]} - Unsupervised: To train an autoencoder we don’t need to do anything fancy, just throw the raw input data at it.
^{[6]} - The aim of an autoencoder is to learn a representation (encoding) for a set of data, typically for dimensionality reduction, by training the network to ignore signal “noise”.
^{[7]} - An autoencoder is a neural network that learns to copy its input to its output.
^{[7]} - The k-sparse autoencoder is based on a linear autoencoder (i.e. with linear activation function) and tied weights.
^{[7]} - In practice, the objective of denoising autoencoders is that of cleaning the corrupted input, or denoising.
^{[7]} - ) Autoencoders are data-specific, which means that they will only be able to compress data similar to what they have been trained on.
^{[8]} - Autoencoders are lossy, which means that the decompressed outputs will be degraded compared to the original inputs (similar to MP3 or JPEG compression).
^{[8]} - Today two interesting practical applications of autoencoders are data denoising (which we feature later in this post), and dimensionality reduction for data visualization.
^{[8]} - As a result, a lot of newcomers to the field absolutely love autoencoders and can't get enough of them.
^{[8]} - A variational autoencoder (VAE) provides a probabilistic manner for describing an observation in latent space, such that the encoder describes a probability distribution for each latent attribute.
^{[9]} - With this bottleneck structure an autoencoder learns to extract the most important information when the input goes through the latent layers.
^{[9]} - Therefore, an autoencoder is an effective way to project data from a high dimension to a lower dimension by extracting the most dominant features and characteristics.
^{[9]} - As mentioned earlier, the synthesis of NMR T2 distributions using VAE-NN requires a two-stage training process prior to the testing and deploying the neural network (Fig. 7.2).
^{[9]} - For more information on autoencoders and other neural network based approaches, see the work of Geoffrey Hinton (Hinton & Salakhutdinov, 2006)3 and many others.
^{[10]} - The autoencoder network has three layers: the input, a hidden layer for encoding, and the output decoding layer.
^{[11]} - Autoencoder networks teach themselves how to compress data from the input layer into a shorter code, and then uncompress that code into whatever format best matches the original input.
^{[11]} - Denoising autoencoder - Using a partially corrupted input to learn how to recover the original undistorted input.
^{[11]} - If you’ve read about unsupervised learning techniques before, you may have come across the term “autoencoder”.
^{[12]} - Autoencoders are one of the primary ways that unsupervised learning models are developed.
^{[12]} - Briefly, autoencoders operate by taking in data, compressing and encoding the data, and then reconstructing the data from the encoding representation.
^{[12]} - Through this process, an autoencoder can learn the important features of the data.
^{[12]} - In addition, the autoencoder is explicitly optimized for the data reconstruction from the code.
^{[13]} - To avoid overfitting and improve the robustness, Denoising Autoencoder (Vincent et al. 2008) proposed a modification to the basic autoencoder.
^{[13]} - Sparse Autoencoder applies a “sparse” constraint on the hidden unit activation to avoid overfitting and improve robustness.
^{[13]} - In \(k\)-Sparse Autoencoder (Makhzani and Frey, 2013), the sparsity is enforced by only keeping the top k highest activations in the bottleneck layer with linear activation function.
^{[13]} - To analyze such data, several machine learning, bioinformatics, and statistical methods have been applied, among them neural networks such as autoencoders.
^{[14]} - In this paper, we investigate several autoencoder architectures that integrate a variety of cancer patient data types (e.g., multi-omics and clinical data).
^{[14]} - In this paper we design and systematically analyze several deep-learning approaches for data integration based on Variational Autoencoders (VAEs) (Kingma and Welling, 2014).
^{[14]} - Autoencoders learn a compressed representation (embedding/code) of the input data by reconstructing it on the output of the network.
^{[14]} - The best performing model was the Composite Model that combined an autoencoder and a future predictor.
^{[15]} - 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 # lstm autoencoder recreate sequence from numpy import array from keras .
^{[15]} - 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 # lstm autoencoder predict sequence from numpy import array from keras .
^{[15]} - 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 # lstm autoencoder reconstruct and predict sequence from numpy import array from keras .
^{[15]} - Processing the benchmark dataset MNIST, a deep autoencoder would use binary transformations after each RBM.
^{[16]} - The decoding half of a deep autoencoder is a feed-forward net with layers 100, 250, 500 and 1000 nodes wide, respectively.
^{[16]} - The decoding half of a deep autoencoder is the part that learns to reconstruct the image.
^{[16]} - The scaled word counts are then fed into a deep-belief network, a stack of restricted Boltzmann machines, which themselves are just a subset of feedforward-backprop autoencoders.
^{[16]} - Generally, you can consider autoencoders as an unsupervised learning technique, since you don’t need explicit labels to train the model on.
^{[17]} - In this tutorial, you’ll learn about autoencoders in deep learning and you will implement a convolutional and denoising autoencoder in Python with Keras.
^{[17]} - The compression in autoencoders is achieved by training the network for a period of time and as it learns it tries to best represent the input image at the bottleneck.
^{[17]} - You feed an image with just five pixel values into the autoencoder which is compressed by the encoder into three pixel values at the bottleneck (middle layer) or latent space.
^{[17]} - Autoencoder is a wildly used deep learning architecture.
^{[18]} - Recent studies focused on modifying the autoencoder algorithm to solve the two challenges.
^{[18]} - The estimation of the model was done by expectation maximization (EM), but it should be easy for autoencoder to do the job, as pointed out that EM=VAE.
^{[18]} - Autoencoder can also be used for supervised learning, similar to principal component regression (PCR).
^{[18]} - After the autoencoder was trained on the training set, we obtained the superset outputs for the training and test sets.
^{[19]} - In the first part of this tutorial, we’ll discuss what autoencoders are, including how convolutional autoencoders can be applied to image data.
^{[20]} - If the goal of an autoencoder is just to reconstruct the input, why even use the network in the first place?
^{[20]} - Later in this tutorial, we’ll be training an autoencoder on the MNIST dataset.
^{[20]} - Autoencoders cannot generate new, realistic data points that could be considered “passable” by humans.
^{[20]} - An autoencoder is a type of artificial neural network used to learn efficient data codings in an unsupervised manner.
^{[21]} - Along with the reduction side, a reconstructing side is also learned, where the autoencoder tries to generate from the reduced encoding a representation as close as possible to its original input.
^{[21]} - Denoising autoencoders create a corrupted copy of the input by introducing some noise.
^{[21]} - This helps to avoid the autoencoders to copy the input to the output without learning features about the data.
^{[21]} - The methodological hierarchy in this work was based on the autoencoder framework, and 12 sampling sizes were considered for landslide susceptibility mapping.
^{[22]} - The final prediction results obtained from the autoencoder modeling were evaluated using the testing data set based on qualitative and quantitative analyses to validate the performance of the models.
^{[22]} - A flowchart of the proposed autoencoder framework is illustrated in Fig.
^{[22]} - The autoencoder is trained to reconstruct input of landslide influencing factors onto the output layer for feature representation, which prevents the simple copying of the data and the network.
^{[22]} - For those getting started with neural networks, autoencoders can look and sound intimidating.
^{[23]} - First and foremost, autoencoders are trained via unsupervised learning, which means you don't need labels.
^{[23]} - An autoencoder is learned to predict its own input by using a noisy version of itself, which forces it to take advantage of structure in the data to learn compact ways of representing it.
^{[23]} - To the best of our knowledge, this research is the first to implement stacked autoencoders by using DAEs and AEs for feature learning in DL.
^{[24]} - Autoencoders are unsupervised neural networks that use machine learning to do this compression for us.
^{[25]} - An autoencoder neural network is an Unsupervised Machine learning algorithm that applies backpropagation, setting the target values to be equal to the inputs.
^{[25]} - Autoencoders are used to reduce the size of our inputs into a smaller representation.
^{[25]} - So you might be thinking why do we need Autoencoders then?
^{[25]} - Configure the VAE to use the specified loss function for the reconstruction, instead of a ReconstructionDistribution.
^{[26]} - Note that this is NOT following the standard VAE design (as per Kingma & Welling), which assumes a probabilistic output - i.e., some p(x|z).
^{[26]} - Set the number of samples per data point (from VAE state Z) used when doing pretraining.
^{[26]} - In this blog we’ve talked about autoencoders several times, both as outliers detection and as dimensionality reduction.
^{[27]} - Now, we present another variation of them, variational autoencoder, which makes possible data augmentation.
^{[27]} - As a kind reminder, an autoencoder network is composed of a pair of two connected networks: an encoder and a decoder.
^{[27]} - But the hidden layer in autoencoders may not be continuous, which might make difficult interpolation.
^{[27]} - The generator takes the form of a fully convolutional autoencoder.
^{[28]} - The autoencoder (left side of diagram) accepts a masked image as an input, and attempts to reconstruct the original unmasked image.
^{[28]} - The discriminator is run using the output of the autoencoder.
^{[28]} - The result is used to influence the cost function used to update the autoencoder's weights.
^{[28]} - We then use autoencoders to reduce the spectra feature dimensions from 1851 to 10 and re-train the ANN models.
^{[29]} - An autoencoder and artificial neural network-based method to estimate parity status of wild mosquitoes from near-infrared spectra.
^{[29]} - We then apply autoencoders to reduce the spectra feature space from 1851 to 10 and re-train ANN models.
^{[29]} - The ANN model achieved an average accuracy of 72% and 93% before and after applying the autoencoder, respectively.
^{[29]}

### 소스

- ↑
^{1.0}^{1.1}^{1.2}^{1.3}Intro to Autoencoders - ↑
^{2.0}^{2.1}^{2.2}^{2.3}Introduction to autoencoders. - ↑
^{3.0}^{3.1}^{3.2}^{3.3}Unsupervised Feature Learning and Deep Learning Tutorial - ↑ 08. 오토인코더 (AutoEncoder)
- ↑
^{5.0}^{5.1}^{5.2}^{5.3}Auto-Encoder: What Is It? And What Is It Used For? (Part 1) - ↑
^{6.0}^{6.1}^{6.2}^{6.3}Applied Deep Learning - Part 3: Autoencoders - ↑
^{7.0}^{7.1}^{7.2}^{7.3}Autoencoder - ↑
^{8.0}^{8.1}^{8.2}^{8.3}Building Autoencoders in Keras - ↑
^{9.0}^{9.1}^{9.2}^{9.3}Autoencoder - an overview - ↑ Autoencoders - an overview
- ↑
^{11.0}^{11.1}^{11.2}Autoencoder - ↑
^{12.0}^{12.1}^{12.2}^{12.3}What is an Autoencoder? - ↑
^{13.0}^{13.1}^{13.2}^{13.3}From Autoencoder to Beta-VAE - ↑
^{14.0}^{14.1}^{14.2}^{14.3}Variational Autoencoders for Cancer Data Integration: Design Principles and Computational Practice - ↑
^{15.0}^{15.1}^{15.2}^{15.3}A Gentle Introduction to LSTM Autoencoders - ↑
^{16.0}^{16.1}^{16.2}^{16.3}Deep Autoencoders - ↑
^{17.0}^{17.1}^{17.2}^{17.3}Keras Autoencoders: Beginner Tutorial - ↑
^{18.0}^{18.1}^{18.2}^{18.3}Autoencoder in biology — review and perspectives - ↑ GSAE: an autoencoder with embedded gene-set nodes for genomics functional characterization
- ↑
^{20.0}^{20.1}^{20.2}^{20.3}Autoencoders with Keras, TensorFlow, and Deep Learning - ↑
^{21.0}^{21.1}^{21.2}^{21.3}Different types of Autoencoders - ↑
^{22.0}^{22.1}^{22.2}^{22.3}The performance of using an autoencoder for prediction and susceptibility assessment of landslides: A case study on landslides triggered by the 2018 Hokkaido Eastern Iburi earthquake in Japan - ↑
^{23.0}^{23.1}^{23.2}Neural Networks 201: All About Autoencoders - ↑ Deep Learning-Based Stacked Denoising and Autoencoder for ECG Heartbeat Classification
- ↑
^{25.0}^{25.1}^{25.2}^{25.3}What are Autoencoders? - ↑
^{26.0}^{26.1}^{26.2}Autoencoders - ↑
^{27.0}^{27.1}^{27.2}^{27.3}Variational autoencoder as a method of data augmentation - ↑
^{28.0}^{28.1}^{28.2}^{28.3}Generative Adversarial Denoising Autoencoder for Face Completion - ↑
^{29.0}^{29.1}^{29.2}^{29.3}An autoencoder and artificial neural network-based method to estimate parity status of wild mosquitoes from near-infrared spectra

## 메타데이터

### 위키데이터

- ID : Q786435

### Spacy 패턴 목록

- [{'LEMMA': 'autoencoder'}]
- [{'LEMMA': 'autoencoder'}]
- [{'LEMMA': 'VAE'}]