Control Systems and Computers

https://doi.org/10.15407/csc.2020.02.041

Control Systems and Computers, 2020, Issue 2 (286), 41-54.

UDK 681.513

Savchenko Yevgeniya A., PhD (Eng.), Senior Research Associate, International Research and Training Center for Information Technologies and Systems NAS and MES of Ukraine, Glushkov ave., 40, Kyiv, 03187, Ukraine, https://orcid.org/0000-0003-4851-9664, savchenko_e@meta.ua

Savchenko Mykhailo Yu., Master, Taras Shevchenko National University of Kyiv, Glushkov ave., 4g, 03022, Kyiv, Ukraine, https://orcid.org/0009-0009-1749-9550, m_savchenko@meta.ua

 The Transfer Learning Task as
the Means of Metalearning Tasks Solution

A review of methods for solving the problems of transfer learning from the point of view of solving the problem of meta-learning. Metalearning is a generalization of all previously solved tasks that allows you to save resources and optimally use the available experience of solving training problems. A scheme for solving the problem of metalearning using the experience of transfer learning is proposed.

 Download full text! (On English)

Keywords: machine learning, transfer learning, modelling, inductive approach, generalization.

1. Transfer learning. [online] Available at: <https://en.wikipedia.org/wiki/Transfer_learning> [Accessed 20 Oct. 2019].
2. Pratt, L. Y.; Thrun, S., 1997. “Machine Learning – Special Issue on Inductive Transfer”, 28 (1), 130 p.
   https://doi.org/10.1023/A:1007322005825
3. The head of Google AI named the main trends in machine learning. [online] Available at: <https://ai-news.ru/2019/12/glava_google_ai_nazval_glavnye_trendy.html> [Accessed 24 Dec. 2019]. (In Russian).
4. Machine Learning. [online] Available at: <https://ru.wikipedia.org/wiki/ Mashinnoye_obucheniye> [Accessed 20 Feb. 2020]. (In Russian).
5. Machine Learning. [online] Available at: <http://www.machinelearning.ru/wiki/index.php?title=Mashinnoye_obucheniye> [Accessed 14 Feb. 2020]. (In Russian).
6. Savchenko, Ye.A., 2019. “The multi-task learning as one of the most important problems in machine learning”. Inductive modeling of complex systems. Zbіrnik of sciences. prats. K.: IRTC ITS, pp. 103-111. (In Ukrainian).
7. Multi-task learning. [online] Available at: <https://en.wikipedia.org/wiki/Multi-task_learning> [Accessed 24 Feb. 2020].
8. Caruana, R., 1997. “Multitask learning,” Machine Learning, 28 (1), pp. 41-75.
   https://doi.org/10.1023/A:1007379606734
9. Vilalta, R., Giraud-Carrier, C., Brazdil, P., Soares, C. “Inductive Transfer”. In book: Encyclopedia of Machine Learning. DOI: 10.1007/978-0-387-30164-8_401.
   https://doi.org/10.1007/978-0-387-30164-8_401
10. Inductive Transfer. [online] Available at: <https://link.springer.com/referenceworkentry/10.1007%2F978-0-387-30164-8_401> [Accessed 14 Feb. 2020].
11. In machine learning, what is the difference between the terms transfer learning, multitask learning, inductive transfer, meta-learning, and learning to learn? [online] Available at: <https://www.quora.com/In-machine-learning-what-is-the-difference-between-the-terms-transfer-learning-multitask-learning-inductive-transfer-meta-learning-and-learning-to-learn> [Accessed 16 Feb. 2020].
12. Pan, S., Yang, Q., 2010. “A Survey on Transfer Learning”. IEEE Transactions on Knowledge and Data Engineering, 22 (10), pp. 1345-1359.
    https://doi.org/10.1109/TKDE.2009.191
13. Silver, D.L., Yang, Q., Li, L. Lifelong Machine Learning Systems: Beyond Learning Algorithms. IEEE transactions on knowledge and data engineering. [online] Available at: <https://www.aaai.org/ocs/index.php/SSS/SSS13/paper/view/5802> [Accessed 20 Feb. 2020].
14. Thrun, S., Pratt, L., (Eds.), 1998. Learning to learn. Norwell, MA, USA: Kluwer Academic Publishers.
    https://doi.org/10.1007/978-1-4615-5529-2
15. Caruana R.A., 1993. “Multitask Learning: A Knowledge-Based Source of Inductive Bias”. Proceedings of the Tenth International Conference “Machine Learning”, University of Massachusetts, Amherst, June 27-29, pp. 41-48.
    https://doi.org/10.1016/B978-1-55860-307-3.50012-5
16. Investigation of the transfer learning ability of convolutional neural networks trained on Imagenet. [online] Available at: <http://www.applied-research.ru/ru/article/view?id=12808> [Accessed 16 Feb. 2020]. (In Russian).
17. Nezhnoye vvedeniye v transfernoye obucheniye dlya glubokogo obucheniya. [online] Available at: <www.machinelearningmastery.ru> [Accessed 20 Feb. 2020]. (In Russian).
18. Google uses transfer training for medical image decryption systems. [online] Available at: <https://www.everest.ua/ru/google-yspolzuet-transfernoe-obuchenye-dlya-system-rasshyfrovky-medyczynskyh-yzobrazhenyj/> [Accessed 16 Jan. 2020]. (In Russian).
19. The state of transfer learning in NLP. [online] Available at: <https://ai-news.ru/2019/08/sostoyanie_transfernogo_obucheniya_v_nlp.html> [Accessed 28 Feb. 2020]. (In Russian).
20. Comprehensive Practical Guide to Learning Translation Using Real-Time Deep Learning Applications. [online] Available at: <https://www.machinelearningmastery.ru/a-comprehensive-hands-on-guide-to-transfer-learning-with-real-world-applications-in-deep-learning-212bf3b2f27a/> [Accessed 1 March 2020]. (In Russian).
21. Handbook of Research on Machine Learning Applications and Trends: Algorithms, Methods and Techniques, 2009. (2 Vol.), 1st Edition. Information Science Reference. 834 p.
22. A Gentle Introduction to Transfer Learning for Deep Learning. [online] Available at: <https://machinelearningmastery.com/transfer-learning-for-deep-learning/> [Accessed 6 March 2020].
23. Kustikova, V.N. Transfer learning deep neural networks. [online] Available at: <http://hpc-education.unn.ru/files/courses/intel-neon-course/Rus/Lectures/Presentations/5_Transfer.pdf> [Accessed 16 Dec. 2019]. (In Russian).
24. Raghu, M., Zhang, Ch., Kleinberg, J., Bengio, S., 2019. “Transfusion: Understanding Transfer Learning for Medical Imaging”. 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada. https://arxiv.org/abs/1902.07208.
25. Sarkar, D., 2018. A Comprehensive Hands-on Guide to Transfer Learning with Real-World Applications in Deep Learning. [online] Available at: <https://towardsdatascience.com/a-comprehensive-hands-on-guide-to-transfer-learning-with-real-world-applications-in-deep-learning-212bf3b2f27a> [Accessed 6 Jan. 2020].
26. How transferable are features in deep neural networks? [online] Available at: <https://arxiv.org/abs/1411.1792> [Accessed 16 Dec. 2019].
27. Stamate, C., Magoulas, G.D., Thomas, M.S.C., 2015. “Transfer learning approach for financial applications”. Neural and Evolutionary Computing, https://arxiv.org/abs/1509.02807.
28. Inductive Transfer of Knowledge: Application of Multi-Task Learning and Feature Net Approaches to Model Tissue-Air Partition Coefficients. [online] Available at: <https://pubs.acs.org/doi/10.1021/ci8002914> [Accessed 20 Feb. 2020].
29. Shalev-Shwart, Sh., Ben-David, Sh., 2014. Understanding Machine Learning. Cambridge University Press, 2018.450 p.
30. Baxter, J., 2000, “A Model of Inductive Bias Learning”. Journal of Artificial Intelligence Research, 12, pp. 149-198.
    https://doi.org/10.1613/jair.731
31. Godfried, I. ICML 2018: Advances in transfer, multitask, and semi-supervised learning. [online] Available at: <https://towardsdatascience.com/icml-2018-advances-in-transfer-multitask-and-semi-supervised-learning-2a15ef7208ec> [Accessed 16 Dec. 2019].
32. Savchenko, Ye., Stepashko, V. Metamodeling and Metalearning Approaches in Inductive Modelling Tools. Preprint. [online] Available at: <https://easychair.org/publications/preprint/6L1W> [Accessed 16 Dec. 2019].
33. Savchenko, Ye., Stepashko, V., 2018. “Metamodeling as a Way to Universalization of Inductive Modeling Tools”. Proceedings of the 13th IEEE International Conference CSIT-2018 & International Workshop on Inductive Modeling, September 11-14, Lviv, Ukraine. Lviv: Publisher “Vezha&Co”, pp. 444-447.
    https://doi.org/10.1109/STC-CSIT.2018.8526582

 Received  10.04.2020