2021년 2월 17일 (수) 08:11에 Pythagoras0님의 편집

2021-02-17T08:11:18Z

Pythagoras0: /* 메타데이터 */ 새 문단

2020-12-26T12:19:14Z

메타데이터: 새 문단

Pythagoras0: /* 노트 */ 새 문단

2020-12-21T10:12:35Z

노트: 새 문단

새 문서

== 노트 ==

===위키데이터===
* ID : [https://www.wikidata.org/wiki/Q6027324 Q6027324]
===말뭉치===
# Transfer learning is a deep learning approach in which a model that has been trained for one task is used as a starting point to train a model for similar task.<ref name="ref_d303c383">[https://kr.mathworks.com/discovery/transfer-learning.html Transfer Learning]</ref>
# Fine-tuning a network with transfer learning is usually much faster and easier than training a network from scratch.<ref name="ref_d303c383" />
# The two commonly used approaches for deep learning are training a model from scratch and transfer learning.<ref name="ref_d303c383" />
# Transfer learning is useful for tasks such object recognition, for which a variety of popular pretrained models, such as AlexNet and GoogLeNet, can be used as a starting point.<ref name="ref_d303c383" />
# We'll take a look at what transfer learning is, how it works, why and when you it should be used.<ref name="ref_8c43ea17">[https://builtin.com/data-science/transfer-learning What is transfer learning? Exploring the popular deep learning approach]</ref>
# Transfer Learning Transfer learning, used in machine learning, is the reuse of a pre-trained model on a new problem.<ref name="ref_8c43ea17" />
# In transfer learning, a machine exploits the knowledge gained from a previous task to improve generalization about another.<ref name="ref_8c43ea17" />
# In transfer learning, the knowledge of an already trained machine learning model is applied to a different but related problem.<ref name="ref_8c43ea17" />
# Transfer learning is the same idea.<ref name="ref_24aa72cf">[https://blogs.nvidia.com/blog/2019/02/07/what-is-transfer-learning/ What Is Transfer Learning?]</ref>
# Recurrent neural networks, often used in speech recognition, can take advantage of transfer learning, as well.<ref name="ref_24aa72cf" />
# In this tutorial, you will learn how to train a convolutional neural network for image classification using transfer learning.<ref name="ref_44ec95fd">[https://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html Transfer Learning for Computer Vision Tutorial — PyTorch Tutorials 1.7.1 documentation]</ref>
# Since we are using transfer learning, we should be able to generalize reasonably well.<ref name="ref_44ec95fd" />
# Transfer learning consists of taking features learned on one problem, and leveraging them on a new, similar problem.<ref name="ref_9da0fac2">[https://keras.io/guides/transfer_learning Transfer learning & fine-tuning]</ref>
# Transfer learning is typically used for tasks when your new dataset has too little data to train a full-scale model from scratch, and in such scenarios data augmentation is very important.<ref name="ref_9da0fac2" />
# To solidify these concepts, let's walk you through a concrete end-to-end transfer learning & fine-tuning example.<ref name="ref_9da0fac2" />
# One answer is transfer learning.<ref name="ref_206c921c">[https://owkin.com/collaborative-ai/transfer-learning/ Transfer Learning and the Rise of Collaborative Artificial Intelligence]</ref>
# Transfer learning is a domain of AI.<ref name="ref_206c921c" />
# It is probably the most used story of transfer learning practice at the moment and one of the hidden reasons why deep learning is such a success.<ref name="ref_206c921c" />
# Indeed, deep learning architecture is very well suited for the transfer learning approach.<ref name="ref_206c921c" />
# This methodology is called transfer learning.<ref name="ref_d68a7d67">[https://datascience.aero/transfer-learning-aviation/ Is Transfer Learning the final step for enabling AI in Aviation?]</ref>
# The key concept behind transfer learning in data science is deep learning models.<ref name="ref_d68a7d67" />
# In addition to being used to improve deep learning models, transfer learning is used in new methodologies for building and training machine learning models in general.<ref name="ref_d68a7d67" />
# The basic idea of transfer learning is then to start with a deep learning network that is pre-initialized from training of a similar problem.<ref name="ref_9170cb47">[https://developer.ibm.com/technologies/artificial-intelligence/articles/transfer-learning-for-deep-learning/ Transfer learning for deep learning]</ref>
# Transfer learning is the method of starting with a pre-trained model and training it for a new — related — problem domain.<ref name="ref_9170cb47" />
# Transfer learning is an important piece of many deep learning applications now and in the future.<ref name="ref_9170cb47" />
# The key to transfer learning is the generality of features within the learning model.<ref name="ref_9170cb47" />
# Following the same approach, a term was introduced Transfer Learning in the field of machine learning.<ref name="ref_846ed3d1">[https://www.geeksforgeeks.org/ml-introduction-to-transfer-learning/ Introduction to Transfer Learning - GeeksforGeeks]</ref>
# When dealing with transfer learning, we come across a phenomenon called freezing of layers.<ref name="ref_846ed3d1" />
# When we use transfer learning in solving a problem, we select a pre-trained model as our base model.<ref name="ref_846ed3d1" />
# Transfer learning is a very effective and fast way, to begin with, a problem.<ref name="ref_846ed3d1" />
# Transfer learning has received attention of data scientists as a methodology for taking advantage of available training data/models from related tasks and applying them to the problem in hand1.<ref name="ref_39abc2f4">[https://www.nature.com/articles/s41598-019-41316-9 Using a Novel Transfer Learning Method for Designing Thin Film Solar Cells with Enhanced Quantum Efficiencies]</ref>
# Example of classification tasks that has benefited from transfer learning include image2,3, web document4,5, brain-computer interface6,7, music8 and emotion9 classification.<ref name="ref_39abc2f4" />
# Despite the above-mentioned applications, transfer learning in optimization problems has not been evaluated thoroughly except a few fields.<ref name="ref_39abc2f4" />
# There are reports of the use of transfer learning in automatic hyper-parameter tuning problems23,24,25,26 to increase training speed and improve prediction accuracy.<ref name="ref_39abc2f4" />
# Transfer learning is a well-established technique for training artificial neural networks (see e.g., Ref.<ref name="ref_c9ee6701">[https://pennylane.ai/qml/demos/tutorial_quantum_transfer_learning.html Quantum transfer learning — PennyLane]</ref>
# We focus on the CQ transfer learning scheme discussed in the previous section and we give a specific example.<ref name="ref_c9ee6701" />
# This is a very small dataset (roughly 250 images), too small for training from scratch a classical or quantum model, however it is enough when using transfer learning approach.<ref name="ref_c9ee6701" />
# We follow the transfer learning approach: First load the classical pre-trained network ResNet18 from the torchvision.models zoo.<ref name="ref_c9ee6701" />
# This paper demonstrates the versatility of this type of regularizer across transfer learning scenarios.<ref name="ref_fb2e9f9b">[https://www.sciencedirect.com/science/article/pii/S0262885619304469 Transfer learning in computer vision tasks: Remember where you come from]</ref>
# Transfer Learning has been utilized by humans since time immemorial.<ref name="ref_9c61f805">[https://www.analyticsinsight.net/transfer-learning-in-deep-learning/ Transfer Learning in Deep Learning]</ref>
# Though this field of transfer learning is relatively new to machine learning, humans have used this inherently in almost every situation.<ref name="ref_9c61f805" />
# We always try to apply the knowledge gained from our past experiences when we face a new problem or task and this is the basis of transfer learning.<ref name="ref_9c61f805" />
# To understand the basic notion of Transfer Learning, consider a model X is successfully trained to perform task A with model M1.<ref name="ref_9c61f805" />
# The authors cover historic methods as well as very recent methods, classifying them into a comprehensive ontology of transfer learning methods.<ref name="ref_ab3e480a">[https://www.cambridge.org/core/books/transfer-learning/CCFFAFE3CDBC245047F1DEC71D9EF3C7 Transfer Learning]</ref>
# Hereafter, successful applications of the shotgun transfer learning in four different scenarios will be described.<ref name="ref_23af4c3c">[https://pubs.acs.org/doi/10.1021/acscentsci.9b00804 Predicting Materials Properties with Little Data Using Shotgun Transfer Learning]</ref>
# We first report a successful application that illustrates the analytic workflow of the transfer learning and some of its potential.<ref name="ref_23af4c3c" />
# Illustrative example of transfer learning for prediction of polymeric C P .<ref name="ref_23af4c3c" />
# The left two panels show prediction performance of a directly supervised random forest and the best transfer learning model using 58 instances of the polymeric C P under 5-fold CV.<ref name="ref_23af4c3c" />
# How do you decide what type of transfer learning you should perform on a new dataset?<ref name="ref_9abec200">[https://cs231n.github.io/transfer-learning/ CS231n Convolutional Neural Networks for Visual Recognition]</ref>
# This form of transfer learning used in deep learning is called inductive transfer.<ref name="ref_ae6f3b6d">[https://machinelearningmastery.com/transfer-learning-for-deep-learning/ A Gentle Introduction to Transfer Learning for Deep Learning]</ref>
# To learn more, visit the Transfer learning guide.<ref name="ref_fde2c1a9">[https://www.tensorflow.org/tutorials/images/transfer_learning Transfer learning and fine-tuning]</ref>
# Over the course of this blog post, I will first contrast transfer learning with machine learning's most pervasive and successful paradigm, supervised learning.<ref name="ref_1c7b7608">[https://ruder.io/transfer-learning/ Transfer Learning - Machine Learning's Next Frontier]</ref>
# I will then outline reasons why transfer learning warrants our attention.<ref name="ref_1c7b7608" />
# Subsequently, I will give a more technical definition and detail different transfer learning scenarios.<ref name="ref_1c7b7608" />
# I will then provide examples of applications of transfer learning before delving into practical methods that can be used to transfer knowledge.<ref name="ref_1c7b7608" />
# In this example, we will see how each of these classifiers can be implemented in a transfer learning solution for image classification.<ref name="ref_cf3b4ddc">[https://towardsdatascience.com/transfer-learning-from-pre-trained-models-f2393f124751 Transfer learning from pre-trained models]</ref>
# The three transfer categories discussed in the previous section outline different settings where transfer learning can be applied, and studied in detail.<ref name="ref_0ee2dbcf">[https://towardsdatascience.com/a-comprehensive-hands-on-guide-to-transfer-learning-with-real-world-applications-in-deep-learning-212bf3b2f27a A Comprehensive Hands-on Guide to Transfer Learning with Real-World Applications in Deep Learning]</ref>
# In case of inductive transfer, modifications such as AdaBoost by Dai and their co-authors help utilize training instances from the source domain for improvements in the target task.<ref name="ref_0ee2dbcf" />
# Inductive transfer techniques utilize the inductive biases of the source task to assist the target task.<ref name="ref_0ee2dbcf" />
# These pre-trained networks/models form the basis of transfer learning in the context of deep learning, or what I like to call ‘deep transfer learning’.<ref name="ref_0ee2dbcf" />
# In 1976 Stevo Bozinovski and Ante Fulgosi published a paper explicitly addressing transfer learning in neural networks training.<ref name="ref_3bba5fa0">[https://en.wikipedia.org/wiki/Transfer_learning Transfer learning]</ref>
# The paper gives a mathematical and geometrical model of transfer learning.<ref name="ref_3bba5fa0" />
# In 1981 a report was given on application of transfer learning in training a neural network on a dataset of images representing letters of computer terminals.<ref name="ref_3bba5fa0" />
# Both positive and negative transfer learning was experimentally demonstrated.<ref name="ref_3bba5fa0" />
# Combined with the idea of transfer learning, the problem of label-free transfer in the target domain was solved.<ref name="ref_17b3f591">[https://www.mdpi.com/2076-3417/10/7/2361/htm Transfer Learning Strategies for Deep Learning-based PHM Algorithms]</ref>
# In Section 2 , the background information of transfer learning is outlined and the transfer scenarios are defined according to the data situation of the target domain and the source domain.<ref name="ref_17b3f591" />
# Transfer learning is a research problem in machine learning that focuses on storing knowledge gained while solving one problem and applying it to a different but related problem.<ref name="ref_cee180eb">[https://alexmoltzau.medium.com/what-is-transfer-learning-6ebb03be77ee What is Transfer Learning?]</ref>
# I stil have not made up my mind, but transfer learning is a topic that I will have to pursue further.<ref name="ref_cee180eb" />
# This is where a technique called ‘transfer learning’ comes in.<ref name="ref_4e373ee0">[https://www.thinkautomation.com/eli5/transfer-learning-in-laymans-terms/ Transfer learning in layman’s terms]</ref>
# In transfer learning, you have a source model trained on a specific dataset.<ref name="ref_4e373ee0" />
# Transfer learning means you’re not starting from scratch – thereby speeding up training time.<ref name="ref_4e373ee0" />
# Beyond the observable benefits, perfecting transfer learning techniques could bring us closer to artificial general intelligence (AGI).<ref name="ref_4e373ee0" />
# As described above, the ULMFiT is a three-stage transfer learning process that includes two types of models: language models and classification/regression models.<ref name="ref_2439392e">[https://jcheminf.biomedcentral.com/articles/10.1186/s13321-020-00430-x Inductive transfer learning for molecular activity prediction: Next - Gen QSAR Models with MolPMoFiT]</ref>
# Recall that homogeneous transfer learning is the case where \({\mathcal{X}}_{{\mathcal{S}}} = {\mathcal{X}}_{{\mathcal{T}}}\).<ref name="ref_33f4f99f">[https://journalofbigdata.springeropen.com/articles/10.1186/s40537-016-0043-6 A survey of transfer learning]</ref>
# In a transfer learning environment, there are scenarios where a feature in the source domain may have a different meaning in the target domain.<ref name="ref_33f4f99f" />
# These transfer learning approaches only attempt to correct for marginal distribution differences between domains.<ref name="ref_33f4f99f" />
# All transfer learning approaches perform better than the baseline approaches.<ref name="ref_33f4f99f" />
# We also theoretically analyse the algorithmic stability and generalization bound of L2T, and empirically demonstrate its superiority over several state-of-the-art transfer learning algorithms.<ref name="ref_1b360b69">[http://proceedings.mlr.press/v80/wei18a.html Transfer Learning via Learning to Transfer]</ref>
# Transfer Learning via Learning to Transfer.<ref name="ref_1b360b69" />
# This is where transfer learning comes into play.<ref name="ref_78c4967c">[https://bdtechtalks.com/2019/06/10/what-is-transfer-learning/ What is transfer learning?]</ref>
# Transfer learning doesn’t require huge compute resources.<ref name="ref_78c4967c" />
# When doing transfer learning, AI engineers freeze the first layers of the pretrained neural network.<ref name="ref_78c4967c" />
# Transfer learning wolves many of the problems of training AI models in an efficient and affordable way.<ref name="ref_78c4967c" />
# In this work, we extend transfer learning with semi-supervised learning to exploit unlabeled instances of (novel) categories with no or only a few labeled instances.<ref name="ref_81b8f5c8">[http://papers.nips.cc/paper/5209-transfer-learning-in-a-transductive-setting Paper]</ref>
# Transfer learning is a machine learning method that involves reusing an existing, trained neural network, developed for one task, as the foundation for another task.<ref name="ref_1c8c2b66">[https://analyticsengines.com/2019/11/29/insights-transfer-learning-doing-more-with-much-less/ Transfer Learning – Doing more with (much) less…]</ref>
# The main challenge of transfer learning is to retain the existing knowledge in the model while adapting the model to your own task.<ref name="ref_1c8c2b66" />
# Transfer learning works with neural networks in a way that it does not with the simpler one-layer models such as logistic regression.<ref name="ref_1c8c2b66" />
# Transfer learning works with neural networks as the different layers of the network can be treated differently.<ref name="ref_1c8c2b66" />
# How to use transfer learning to build state-of-the-art customer service AI!<ref name="ref_d4d394fc">[https://www.ultimate.ai/blog/ai-automation/transfer-learning-in-customer-service-automation Transfer Learning in Customer Service Automation]</ref>
# Transfer learning is a method that allows us to use the knowledge gained from other tasks in order to tackle new but similar problems quickly and effectively.<ref name="ref_d4d394fc" />
# Solving the Finnish problem with transfer learning prompted us to develop our architecture to use a single model across all clients and regions.<ref name="ref_d4d394fc" />
# More interestingly, by being able to apply ways of thinking from one task to another, transfer learning unlocks deep learning potential from smaller datasets.<ref name="ref_d4d394fc" />
# Transfer learning, in which a network is trained on one task and re-purposed on another, is often used to produce neural network classifiers when data is scarce or full-scale training is too costly.<ref name="ref_75c4c082">[https://openreview.net/forum?id=ryebG04YvB Adversarially robust transfer learning]</ref>
# We consider robust transfer learning, in which we transfer not only performance but also robustness from a source model to a target domain.<ref name="ref_75c4c082" />
# Recently, transfer learning methods have been applied to reuse knowledge from performance models trained in one environment in another.<ref name="ref_ba4ac57b">[https://www.usenix.org/conference/opml19/presentation/iqbal Transfer Learning for Performance Modeling of Deep Neural Network Systems]</ref>
# In this paper, we perform an empirical study to understand the effectiveness of different transfer learning strategies for building performance models of DNN systems.<ref name="ref_ba4ac57b" />
# Transfer learning is the application of knowledge gained from completing one task to help solve a different, but related, problem.<ref name="ref_160f5e3c">[https://searchcio.techtarget.com/definition/transfer-learning What is transfer learning?]</ref>
# Through transfer learning, methods are developed to transfer knowledge from one or more of these source tasks to improve learning in a related target task.<ref name="ref_160f5e3c" />
# Transfer learning theory During transfer learning, knowledge is leveraged from a source task to improve learning in a new task.<ref name="ref_160f5e3c" />
# Transfer learning is an approach used in machine learning where a model that was created and trained for one task, is reused as the starting point for a secondary task.<ref name="ref_4d8dda78">[https://missinglink.ai/guides/neural-network-concepts/transfer-learning-overview/ Transfer Learning: An Overview]</ref>
# Transfer learning is a widely used technique for improving the performance of neural networks when labeled training data is scarce.<ref name="ref_cbc4fb65">[https://www.amazon.science/blog/when-does-transfer-learning-work When does transfer learning work?]</ref>
# When is transfer learning effective, and when is it not?<ref name="ref_cbc4fb65" />
# And if you’re going to do transfer learning, what task should you use for pretraining?<ref name="ref_cbc4fb65" />
# One of the settings we considered was that of meta-transfer learning, which is a combination of transfer learning and meta-learning.<ref name="ref_cbc4fb65" />
# Transfer learning reduces the size of a training dataset by utilizing the knowledge in a pre-trained neural network.<ref name="ref_fb804ccd">[https://www.nature.com/articles/s41598-020-64165-3 Stepwise PathNet: a layer-by-layer knowledge-selection-based transfer learning algorithm]</ref>
# In some transfer learning cases, the pre-trained neural network for the source task has been trained by a large computer.<ref name="ref_fb804ccd" />
===소스===
<references />

@@ 113번째 줄: / 113번째 줄: @@
   <references />
-== 메타데이터 ==
+==메타데이터==
 ===위키데이터===
 * ID :  [https://www.wikidata.org/wiki/Q6027324 Q6027324]

← 이전 판		2020년 12월 26일 (토) 12:19 판
112번째 줄:		112번째 줄:
	===소스===		===소스===
	<references />		<references />
		+
		+	== 메타데이터 ==
		+
		+	===위키데이터===
		+	* ID : [https://www.wikidata.org/wiki/Q6027324 Q6027324]

Inductive transfer - 편집 역사

2021년 2월 17일 (수) 08:11에 Pythagoras0님의 편집

Pythagoras0: /* 메타데이터 */ 새 문단

Pythagoras0: /* 노트 */ 새 문단