Control Systems and Computers, N3, 2020, Article 2

Control Systems and Computers, 2020, Issue 3 (287), pp. 15-27.

UDK  004.852

O.G. RUDENKO, Doctor (Eng.), Professor, Head of Department of Сomputer intelligent technologies and systems, Kharkiv National University of Radio Electronics, Nauky Ave. 14, Kharkiv, 61166, Ukraine,

O.O. BEZSONOV, Doctor (Eng.), Professor, Department of Сomputer intelligent technologies and systems, Kharkiv National University of Radio Electronics, Nauky Ave. 14, Kharkiv, 61166, Ukraine,

ADALINE Robust Multistep Training Algorithm

The article considers the multi-step ADALINE training algorithm when using the correntropy information criterion as a learning criterion, determines the conditions for the convergence of the algorithm, and shows that in the steady state the resulting estimate is unbiased. The importance of choosing the width of the Gaussian core, which affects the convergence rate of the estimation algorithms and the error in the steady state, is noted, and the feasibility of developing procedures for adaptive correction of the core width is indicated.

Download full text! (On English)

Keywords: validation, learning outcomes, non-formal learning, informal learning, professions, competencies, skills, ontology, ESCO, Semantic Web.

  1. Widrow, B., Hoff, M., 1960. “Adaptive switching circuits”, IRE WESCON Convention Record. Part 4. New York: Institute of Radio Engineers, pp. 96–104.
  2. Kaczmarz, S., 1937. Angenäherle Auflösung von Systemen linearer Gleichungen, Bull. Int. Acad. Polon. Sci. Lett., C 1, Sci. Math. Nat., Ser. A, pp. 355-357. [In Eng.: Kaczmarz, S., 1993. “Approximate solution of systems of linear equations”, Int. J. of Control, 57, pp. 1269-1271].
  3. Liberal, V.D., Rudenko, O.G., Bezsonov, A.A., 2018. “Study of convergence of one-step adaptive identification algorithms”. Journal of Automation and Information Sciences, 50 (10), pp. 60-76.
  4. Rudenko, O.G., Bezsonov, O.O., “The Regularized Adaline Learning Algorithm for the Problem of Evaluation of Non-Stationary Parameters”, Control Systems and Computers, 2019, 1, pp.22-30. (In Russian).
  5. Huber, P., 1984. Robustness in statistics. M.: Mir, 304 p.
  6. Hampel, F.R., Ronchetti, E.M., Rousseeuw, P.J., Stahel, W.A., Robust Statistics. The Approach Based on Influence Functions. N.Y.: John Wiley and Sons, 1986, 526 p.
  7. Rudenko, O.G., Bezsonov, A.A., 2010.“Robust learning wavelet neural networks”, Journal of Automation and Information Sciences, 42(10), pp. 1-15.
  8. Rudenko, O., Bezsonov, O., 2011. “Function approximation using robust radial basis function networks”, J. of Intelligent Learning Systems and Applications, 3, pp.17-25.
  9. Rudenko, O.G., Bezsonov, A.A., 2012. Radial basic networks m-training by asymmetric influence functions”.Journal of Automation and Information Sciences, 44(2), pp. 48-64.
  10. Walach, E. Widrow, D., 1984. “The least mean fourth (LMF) adaptive algorithm and it’s family”, IEEE Trans, IT 30, pp. 275–283.
  11. Chambers, J., Avlonitis, A., 1997. “A Robust Mixed-Norm Adaptive Filter Algorithm”, IEEE Signal Processing Letters, 4 (2), pp. 46-48.
  12. Papoulis, V., Stathaki, T., 2004. “A Normalized Robust Mixed-Norm Adaptive Algorithm for System Identification”, IEEE Signal Processing Letters, 11 (1), pp. 56-59
  13. Chambers, J., Tanrikulu, O, Constantinides, G., 1984. Least mean mixed-norm adaptive filtering, Electronics Letters, 30 (19), pp. 1574-1575.
  14. Zerguine, A., 2012. A variable-parameter normalized mixed-norm (VPNMN) adaptive algorithm, EURASIP Journal on Advances in Signal Processing, 55, 13 p.
  15. Rudenko, O.G., Bezsonov, O.O., Serdyuk, N.M. Olyynik, K.O., Romanyuk, O.S., 2019. Robastna identyfikatsiya obʺyektiv za nayavnostyu nehausivsʹkykh zavad”, Byonyka yntellekta, Bionics of Intellect, 2(93), pp. 7-12. (In Ukrainian).
  16. Rudenko, O.G., Bezsonov, O.O., Serdyuk, N.M. Olyynik, K.O., Romanyuk, O.S., 2020. Robastna identyfikatsiya obʺyektiv na osnovi minimizatsiyi kombinovanoho funktsionalu, Systemy obrobky informatsiyi, 2020,1 (160), pp.80-88. (In Ukrainian).
  17. Principe, C., Xu, D., Zhao, Q., Fisher, J. W., 2000. “Learning from examples with information theoretic criteria”, J. VLSI Signal Process. Syst., 26 (1–2), pp. 61-77.
  18. Principe, C., Xu, D., Fisher, J., 2000. Information-theoretic learning”. Unsupervised Adaptive Filtering. New York: Wiley, pp. 265–319.
  19. Chen, B., Hu, J., Pu, L., Sun, Z., 2007. “Stochastic gradient algorithm under (h, φ)-entropy criterion”, Circuits Syst. Signal Process, 26, pp. 941–960.
  20. Santamaria, I., Pokharel, P.P, Jose, C., Principe, J.C., 2006. “Generalized Correlation Function: Definition, Properties, and Application to Blind Equalization”, IEEE Trans. on Signal Processing, 54, 6, pp. 2187-2197.
  21. Liu, W., Pokharel, P.P, Principe, J.C., 2007. “Correntropy: Properties and Applications in Non-Gaussian Signal Processing”, IEEE Trans. on Signal Processing, 1, pp. 5286-5298.
  22. Wang, W., Zhao, J., H. Qu, H., Chen, B., Principe, J.C., 2015. “An adaptive kernel width update method of correntropy for channel estimation”, IEEE International Conference on Digital Signal Processing (DSP), pp. 916–920.
  23. Chen, B., Xing, L., Liang, J., Zheng, N, Principe, J.C., 2014. Steady-state mean-square error analysis for adaptive filtering under the maximum correntropy criterion”, Signal Process. Lett. IEEE, 21(7), pp. 880–884.
  24. Ma, W., Qua, H., Guib, G., Li, Xu L., Zhao, J., Chen, B., 2015. “Maximum correntropy criterion based sparse adaptive filtering algorithms for robust channel estimation under non-Gaussian environments”, J. of the Franklin Institute, 352 (2), pp. 2708-2727.
  25. Guo, Y., Ma, B, Li, Y., 2016. Kernel-Width Adaption Diffusion Maximum Correntropy Algorithm”, IEEE Acces, 4, pp.1-14.
  26. Lu, L., Zhao, H., 2017. “Active impulsive noise control using maximum correntropy with adaptive kernel size”, Mechanical Systems and Signal Processing, 87, Part A, pp. 180-191.
  27. Qi, Y., Wang, Y., Zhang, J., Zhu, J., Zheng, X., 2014. Robust Deep Network with Maximum Correntropy Criterion for Seizure Detection, BioMed Research International, Article ID 703816, 10 p.
  28. Huang, F., Zhang, J., Zhang, S., 2017. “Adaptive filtering under a variable kernel width maximum correntropy criterion”, IEEE Trans. on Circuits and Systems II: Express Briefs, 64 (10), pp. 1247–1251.
  29. PerelmanI., 1982. Operativnaya identifikatsiya ob”yektov upravleniya. M.: Energoizdat, 272 p. (In Russian).
  30. Rudenko, O.G., Terenkovskiy, I.D., Shtefan, A., Oda, G.A., 1998. “Modifitsirovannyy algoritm tekushchego regressionnogo analiza v zadachakh identifikatsii i prognozirovaniya”, Radioelektronika i informatika, 4(05), pp.58-61. (In Russian).
  31. Goodwin,  G., Sin, K.S., 2014. Adaptive filtering prediction and control. N.Y.: Dover Publications, 560 p.
  32. Streng, G., 1980. Lineynaya algebra i yeye primeneniya. M.: Mir, 1980, 454 p.
  33. Karchevskiy, Ye.M., Karchevskiy, M.M., 2018. Lektsii po lineynoy algebre i analiticheskoy geometrii: uchebnoye posobiye. Kazan’: Izd-vo Kazanskogo un-ta, 426 p. (In Russian).
  34. Doyle, C., Stein, G., 1981. “Multivariable Feedback Design: Concepts for a Classical/Modern Synthesis”, IEEE Trans. on Automatic Control, 1981, 26, 1, pp. 4-16.
  35. Zhao,  S., B. Chen, B., Principe, J.C., 2012. “An adaptive kernel width update for correntropy”, Proc. of the International Joint Conference on Neural Networks (IJCNN ’12), Brisbane, Australia, pp. 1–5.
  36. Jones, C., Marron, J.S., Sheather, S.J., 1996. “A brief survey of bandwidth selection for density estimation”, Journal of the American Statistical Association, 91, 433, pp. 401–407.
  37. Silverman, W., 1986. Density Estimation for Statistics and Data Analysis, vol. 3: CRC Press: New York, NY, USA, 176 p.
  38. Bowman, W., 1984. “An alternative method of cross-validation for the smoothing of density estimates”, Biometrika, 71 (2), pp. 353–360.
  39. ScotW., Terrell, G.R., 1987. “Biased and unbiased cross-validation in density estimation”, Journal of the American Statistical Association, 82, 400, pp. 1131–1146.
  40. Shi, L., Zhao, H., Zakharov, Y., 2018. An Improved Variable Kernel Width for Maximum Correntropy Criterion Algorithm, IEEE Trans. on Circuits and Systems II: Express Briefs, 5p.

Received 14.06.2020.