Transfer learning is the way in which humans apply their knowledge in a task to learn another task. Transfer learning gains the knowledge from one or more tasks that were successfully approved and applies this knowledge to solve the new problem. In Transfer learning, the distributions and the data domains used for training and testing can be different. Transfer learning supports two types of learning:
- Positive Transfer: When learning in one situation facilitates learning in another situation is known as Positive Transfer.
- Negative Transfer: When learning of one task makes the learning of another task harder is known as Negative Transfer.
Advantages of Transfer Learning:
- Transfer learning has a wide area of research on larger applications, such as social network analysis and network classification.
- Transfer learning saves both cost and time and enables us to develop many similar applications related to certain concepts. For example, if we know to play the recorder, we may apply this knowledge of noting the tracks of music to learn to play the piano.
- Transfer learning is useful in scenarios where the data become outdated over a particular time period or when the data is dynamically changing. For example, where the transfer learning is applied, is web-document classification, where we may have trained the model with categorical data of different newsgroups. If we use the web data on the websites to train the classifier can easily become outdated because the data of the topics can be changed on the websites frequently.
- Email spam filtering is another application for transfer learning. We could filter the emails of users as spam or ham by training the model with classification algorithms. If new emails come along and those emails can be different from the originally trained emails, hence we need to adapt the learned model to be able to classify the new emails of different categories.
- Knowledge transfer through Transfer learning would reduce the need to annotate huge data, This is the core advantage of transfer learning.
- Thus, transfer learning saves time and reduces the effort to rebuild the designs and models from scratch.
Approaches of Transfer Learning:
There are different approaches to inculcate transfer learning in data mining. The most common approach is the TrAdaBoost (Transfer AdaBoost) algorithm and this is an instance-based transfer learning approach. In this approach, some of the trained data can be adjusted and reweighted the class labels data and used it to learn the target task. Consider the above-discussed example of web-document classification, where the distribution of the trained classifier of old data is different from the target data. TrAdaBoost assumes that both the training data and testing data(target domain data) are having the same set of domains of attributes and they also have the same set of class labels. But it becomes difficult if the distribution of the train and test data are of different domains. TrAdaBoost also configures the properties of the AdaBoost ensemble method. The old source data or the training data can be further useful as TrAdaBoost assumes that most of it can be useful in training the other new classification model. We can do that by filtering out the old data which is different from the new data and we need to adjust the weights assigned to the training tuples.
Transfer learning becomes a positive transfer if the model for the new problem is successfully built from the base knowledge patterns. Negative transfer occurs if the newly trained design model is so far irrelative compared to the base model and heterogeneous to the target data. Negative transfer occurs if the training data is not preprocessed or if the features of the base model are not suitable to the particular data attributes.
Challenges of Transfer Learning:
- One of the challenges for transfer learning is Negative transfer. This makes the learning of the models inefficient. So it is necessary to avoid Negative transfer.
- If the data is not preprocessed then the Transfer learning may provide irrelevant models.