2021년 2월 17일 (수) 07:41에 Pythagoras0님의 편집

2021-02-17T07:41:45Z

Pythagoras0: /* 메타데이터 */ 새 문단

2020-12-26T12:01:18Z

메타데이터: 새 문단

Pythagoras0: /* 노트 */ 새 문단

2020-12-23T07:49:27Z

노트: 새 문단

새 문서

== 노트 ==

===위키데이터===
* ID : [https://www.wikidata.org/wiki/Q798503 Q798503]
===말뭉치===
# There is no shortage of papers online that attempt to explain how backpropagation works, but few that include an example with actual numbers.<ref name="ref_12326178">[https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/ A Step by Step Backpropagation Example]</ref>
# The backpropagation algorithm was originally introduced in the 1970s, but its importance wasn't fully appreciated until a famous 1986 paper by David Rumelhart, Geoffrey Hinton, and Ronald Williams.<ref name="ref_ebf3a9bb">[http://neuralnetworksanddeeplearning.com/chap2.html Neural networks and deep learning]</ref>
# If you're not crazy about mathematics you may be tempted to skip the chapter, and to treat backpropagation as a black box whose details you're willing to ignore.<ref name="ref_ebf3a9bb" />
# And so backpropagation isn't just a fast algorithm for learning.<ref name="ref_ebf3a9bb" />
# I've written the rest of the book to be accessible even if you treat backpropagation as a black box.<ref name="ref_ebf3a9bb" />
# Many neural network books (Haykin, 1994; Bishop, 1995; Ripley, 1996) do not formulate backpropagation in vector-matrix terms.<ref name="ref_2a0be238">[https://www.sciencedirect.com/topics/computer-science/backpropagation-algorithm Backpropagation Algorithm - an overview]</ref>
# Hinton and Salakhutdinov (2006) noted that it has been known since the 1980s that deep autoencoders, optimized through backpropagation, could be effective for nonlinear dimensionality reduction.<ref name="ref_2a0be238" />
# In this chapter we discuss a popular learning method capable of handling such large learning problems—the backpropagation algorithm.<ref name="ref_2ebda9af">[https://link.springer.com/chapter/10.1007/978-3-642-61068-4_7 The Backpropagation Algorithm]</ref>
# In other words, backpropagation aims to minimize the cost function by adjusting network’s weights and biases.<ref name="ref_616ab8c4">[https://towardsdatascience.com/understanding-backpropagation-algorithm-7bb3aa2f95fd Understanding Backpropagation Algorithm]</ref>
# One way to train our model is called as Backpropagation.<ref name="ref_ee2d04fd">[https://www.edureka.co/blog/backpropagation/ Training A Neural Network]</ref>
# The Backpropagation algorithm looks for the minimum value of the error function in weight space using a technique called the delta rule or gradient descent.<ref name="ref_ee2d04fd" />
# The structure of a BP network is shown in Figure 12.4.<ref name="ref_60db9508">[https://www.sciencedirect.com/topics/engineering/backpropagation-algorithm Backpropagation Algorithm - an overview]</ref>
# The BP network is known from the abbreviation.<ref name="ref_60db9508" />
# The BP algorithm can be summarized by the steps below: (1) Initialize all weightings and thresholds.<ref name="ref_60db9508" />
# The project describes teaching process of multi-layer neural network employing backpropagation algorithm.<ref name="ref_263b89d5">[http://home.agh.edu.pl/~vlsi/AI/backp_t_en/backprop.html Backpropagation]</ref>
# Only in the middle eighties the backpropagation algorithm has been worked out.<ref name="ref_263b89d5" />
# It is one kind of backpropagation network which produces a mapping of a static input for static output.<ref name="ref_78df0a58">[https://www.guru99.com/backpropogation-neural-network.html Back Propagation Neural Network: Explained With Simple Example]</ref>
# Recurrent backpropagation is fed forward until a fixed value is achieved.<ref name="ref_78df0a58" />
# Backpropagation is an algorithm commonly used to train neural networks.<ref name="ref_afb68f78">[https://missinglink.ai/guides/neural-network-concepts/backpropagation-neural-networks-process-examples-code-minus-math/ Backpropagation in Neural Networks: Process, Example & Code]</ref>
# Backpropagation is simply an algorithm which performs a highly efficient search for the optimal weight values, using the gradient descent technique.<ref name="ref_afb68f78" />
# We’ll explain the backpropagation process in the abstract, with very simple math.<ref name="ref_afb68f78" />
# The backpropagation algorithm calculates how much the final output values, o1 and o2, are affected by each of the weights.<ref name="ref_afb68f78" />
# Generalizations of backpropagation exists for other artificial neural networks (ANNs), and for functions generally.<ref name="ref_6b8aa611">[https://en.wikipedia.org/wiki/Backpropagation Backpropagation]</ref>
# Backpropagation computes the gradient in weight space of a feedforward neural network, with respect to a loss function.<ref name="ref_6b8aa611" />
# In the derivation of backpropagation, other intermediate quantities are used; they are introduced as needed below.<ref name="ref_6b8aa611" />
# This is the reason why backpropagation requires the activation function to be differentiable.<ref name="ref_6b8aa611" />
# Thus, for the purposes of derivation, the backpropagation algorithm will concern itself with only one input-output pair.<ref name="ref_8795944e">[https://brilliant.org/wiki/backpropagation/ Brilliant Math & Science Wiki]</ref>
# This equation is where backpropagation gets its name.<ref name="ref_8795944e" />
# The principle of the backpropagation approach is to model a given function by modifying internal weightings of input signals to produce an expected output signal.<ref name="ref_38d3d640">[https://machinelearningmastery.com/implement-backpropagation-algorithm-scratch-python/ How to Code a Neural Network with Backpropagation In Python (from scratch)]</ref>
# Technically, the backpropagation algorithm is a method for training the weights in a multilayer feed-forward neural network.<ref name="ref_38d3d640" />
# Running the example prints the network after the backpropagation of error is complete.<ref name="ref_38d3d640" />
# However, the main learning mechanism behind these advances – error backpropagation – appears to be at odds with neurobiology.<ref name="ref_deb80be4">[https://papers.nips.cc/paper/8089-dendritic-cortical-microcircuits-approximate-the-backpropagation-algorithm Paper]</ref>
# We demonstrate the learning capabilities of the model in regression and classification tasks, and show analytically that it approximates the error backpropagation algorithm.<ref name="ref_deb80be4" />
# Examining the algorithm you can see why it's called backpropagation.<ref name="ref_284021ee">[https://eng.libretexts.org/Bookshelves/Computer_Science/Book%3A_Neural_Networks_and_Deep_Learning_(Nielsen)/02%3A_How_the_backpropagation_algorithm_works/2.03%3A_The_backpropagation_algorithm 2.3: The backpropagation algorithm]</ref>
# Backpropagation with linear neurons Suppose we replace the usual non-linear \(σ\) function with \(σ(z)=z\) throughout the network.<ref name="ref_284021ee" />
# Rewrite the backpropagation algorithm for this case.<ref name="ref_284021ee" />
# As I've described it above, the backpropagation algorithm computes the gradient of the cost function for a single training example, \(C=C_x\).<ref name="ref_284021ee" />
# What if I told you those people don’t even know what machine learning and things like backpropagation really are?<ref name="ref_7263d7ec">[https://www.kdnuggets.com/2019/01/backpropagation-algorithm-demystified.html The Backpropagation Algorithm Demystified]</ref>
# Now that you’ve learnt some of the main principles of Backpropagation in Machine Learning you understand how it isn’t about having technology come to life so they can abolish the human race.<ref name="ref_7263d7ec" />
# Backpropagation allows us to calculate the gradient of the loss function with respect to each of the weights of the network.<ref name="ref_1cc8ecc5">[https://deepai.org/machine-learning-glossary-and-terms/backpropagation Backpropagation]</ref>
# Backpropagation involves the calculation of the gradient proceeding backwards through the feedforward network from the last layer through to the first.<ref name="ref_1cc8ecc5" />
# The backpropagation algorithm involves first calculating the derivates at layer N, that is the last layer.<ref name="ref_1cc8ecc5" />
# Initially, the network was trained using backpropagation through all the 18 layers.<ref name="ref_1cc8ecc5" />
# Backpropagation (backward propagation) is an important mathematical tool for improving the accuracy of predictions in data mining and machine learning.<ref name="ref_ecd1ed02">[https://searchenterpriseai.techtarget.com/definition/backpropagation-algorithm What is backpropagation algorithm?]</ref>
# Artificial neural networks use backpropagation as a learning algorithm to compute a gradient descent with respect to weights.<ref name="ref_ecd1ed02" />
# Because backpropagation requires a known, desired output for each input value in order to calculate the loss function gradient, it is usually classified as a type of supervised machine learning.<ref name="ref_ecd1ed02" />
# Professor Geoffrey Hinton explains backpropagation.<ref name="ref_ecd1ed02" />
# We can define the backpropagation algorithm as an algorithm that trains some given feed-forward Neural Network for a given input pattern where the classifications are known to us.<ref name="ref_96cd4b8d">[https://www.mygreatlearning.com/blog/backpropagation-algorithm/ An Introduction to Backpropagation Algorithm and How it Works?]</ref>
# Before we deep dive into backpropagation, we should be aware about who introduced this concept and when.<ref name="ref_96cd4b8d" />
# Today, backpropagation is doing good.<ref name="ref_96cd4b8d" />
# Neural network training happens through backpropagation.<ref name="ref_96cd4b8d" />
# Computing Gradients (Part 2)" we will go over the actual backpropagation and see step by step how does the math work.<ref name="ref_ddc2607d">[https://www.linkedin.com/pulse/understanding-backpropagation-algorithm-introducing-math-kostadinov/ Understanding Backpropagation algorithm: Introducing the math behind neural networks (Part 1)]</ref>
# Backpropagation is the central mechanism by which neural networks learn.<ref name="ref_9a0078e6">[https://wiki.pathmind.com/backpropagation A Beginner's Guide to Backpropagation in Neural Networks]</ref>
# Backpropagation takes the error associated with a wrong guess by a neural network, and uses that error to adjust the neural network’s parameters in the direction of less error.<ref name="ref_9a0078e6" />
# Backpropagation works by approximating the non-linear relationship between the input and the output by adjusting the weight values internally.<ref name="ref_4c6214e7">[https://www.cse.unsw.edu.au/~cs9417ml/MLP2/BackPropagation.html Mutli-Layer Perceptron]</ref>
# The following figure shows the topology of the Backpropagation neural network that includes and input layer, one hidden layer and an output layer.<ref name="ref_4c6214e7" />
# The operations of the Backpropagation neural networks can be divided into two steps: feedforward and Backpropagation.<ref name="ref_4c6214e7" />
# Some modifications to the Backpropagation algorithm allows the learning rate to decrease from a large value during the learning process.<ref name="ref_4c6214e7" />
# The backpropagation algorithm — the process of training a neural network — was a glaring one for both of us in particular.<ref name="ref_b97f023b">[https://ayearofai.com/rohan-lenny-1-neural-networks-the-backpropagation-algorithm-explained-abf4609d4f9d Rohan & Lenny #1: Neural Networks & The Backpropagation Algorithm, Explained]</ref>
# Together, we embarked on mastering backprop through some great online lectures from professors at MIT & Stanford.<ref name="ref_b97f023b" />
# Today, we’ll do our best to explain backpropagation and neural networks from the beginning.<ref name="ref_b97f023b" />
# The backpropagation algorithm was a major milestone in machine learning because, before it was discovered, optimization methods were extremely unsatisfactory.<ref name="ref_b97f023b" />
# In the next section, I'll introduce a way to visualize the process we've just developed in addition to presenting an end-to-end method for implementing backpropagation.<ref name="ref_2474e33d">[https://www.jeremyjordan.me/neural-networks-training/ Neural networks: training with backpropagation.]</ref>
# Note: Backpropagation is simply a method for calculating the partial derivative of the cost function with respect to all of the parameters.<ref name="ref_2474e33d" />
# The first way to do backpropagation is to backpropagate through a non linear function.<ref name="ref_b71a2972">[https://atcold.github.io/pytorch-Deep-Learning/en/week02/02-1/ Introduction to Gradient Descent and Backpropagation Algorithm · Deep Learning]</ref>
# For the backprop algorithm, we need two sets of gradients - one with respect to the states (each module of the network) and one with respect to the weights (all the parameters in a particular module).<ref name="ref_b71a2972" />
# We can again use chain rule for backprop.<ref name="ref_b71a2972" />
# The proposed algorithms improve the backpropagation training in terms of both convergence rate and convergence characteristics, such as stable learning and robustness to oscillations.<ref name="ref_c183aa6d">[https://www.mitpressjournals.org/doi/10.1162/089976699300016223 Improving the Convergence of the Backpropagation Algorithm Using Learning Rate Adaptation Methods]</ref>
# In algorithmic modification, the Standard BP algorithm(SBP) was modified by introducing momentum term, Quasi Newton method as a second order method, and Resillent backpropation algorithm.<ref name="ref_8f512bcc">[https://content.iospress.com/articles/journal-of-intelligent-and-fuzzy-systems/ifs190063 An improved third term backpropagation algorithm – inertia expanded chebyshev orthogonal polynomial]</ref>
# The proposed method shows better performance when compared to standard backpropagation algorithm (SBP) and backpropagation algorithm with momentum (SBPM).<ref name="ref_8f512bcc" />
# Specifically, explanation of the backpropagation algorithm was skipped.<ref name="ref_e5fc12a3">[https://rubikscode.net/2018/01/22/backpropagation-algorithm-in-artificial-neural-networks/ Backpropagation Algorithm in Artificial Neural Networks]</ref>
# Like the majority of important aspects of Neural Networks, we can find roots of backpropagation in the 70s of the last century.<ref name="ref_e5fc12a3" />
# One of the main tasks of backpropagation is to give us information on how quickly the error changes when weights are changed.<ref name="ref_e5fc12a3" />
# As mentioned, there are some assumptions that we need to make regarding this function in order for backpropagation to be applicable.<ref name="ref_e5fc12a3" />
# Refer to the figure 2.12 that illustrates the backpropagation multilayer network with layers.<ref name="ref_57e74518">[http://wwwold.ece.utep.edu/research/webfuzzy/docs/kk-thesis/kk-thesis-html/node22.html 2.4.4 Backpropagation Learning Algorithm]</ref>
===소스===
<references />

@@ 82번째 줄: / 82번째 줄: @@
   <references />
-== 메타데이터 ==
+==메타데이터==
 ===위키데이터===
 * ID :  [https://www.wikidata.org/wiki/Q798503 Q798503]

← 이전 판		2020년 12월 26일 (토) 12:01 판
81번째 줄:		81번째 줄:
	===소스===		===소스===
	<references />		<references />
		+
		+	== 메타데이터 ==
		+
		+	===위키데이터===
		+	* ID : [https://www.wikidata.org/wiki/Q798503 Q798503]

오차역전파법 - 편집 역사

2021년 2월 17일 (수) 07:41에 Pythagoras0님의 편집

Pythagoras0: /* 메타데이터 */ 새 문단

Pythagoras0: /* 노트 */ 새 문단