Control Systems and Computers

https://doi.org/10.15407/csc.2020.03.015

Control Systems and Computers, 2020, Issue 3 (287), pp. 15-27.

UDK  004.852

O.G. RUDENKO, Doctor (Eng.), Professor, Head of Department of Сomputer intelligent technologies and systems, Kharkiv National University of Radio Electronics, Nauky Ave. 14, Kharkiv, 61166, Ukraine, 
oleg.rudenko@hneu.net

O.O. BEZSONOV, Doctor (Eng.), Professor, Department of Сomputer intelligent technologies and systems, Kharkiv National University of Radio Electronics, Nauky Ave. 14, Kharkiv, 61166, Ukraine, oleksandr.bezsonov@hneu.net

ADALINE Robust Multistep Training Algorithm

The article considers the multi-step ADALINE training algorithm when using the correntropy information criterion as a learning criterion, determines the conditions for the convergence of the algorithm, and shows that in the steady state the resulting estimate is unbiased. The importance of choosing the width of the Gaussian core, which affects the convergence rate of the estimation algorithms and the error in the steady state, is noted, and the feasibility of developing procedures for adaptive correction of the core width is indicated.

Download full text! (On English)

Keywords: validation, learning outcomes, non-formal learning, informal learning, professions, competencies, skills, ontology, ESCO, Semantic Web.

1. Widrow, B., Hoff, M., 1960. “Adaptive switching circuits”, IRE WESCON Convention Record. Part 4. New York: Institute of Radio Engineers, pp. 96-104.
   https://doi.org/10.21236/AD0241531
2. Kaczmarz, S., 1937. Angenäherle Auflösung von Systemen linearer Gleichungen, Bull. Int. Acad. Polon. Sci. Lett., C 1, Sci. Math. Nat., Ser. A, pp. 355-357. [In Eng.: Kaczmarz, S., 1993. “Approximate solution of systems of linear equations”, Int. J. of Control, 57, pp. 1269-1271].
3. Liberal, V.D., Rudenko, O.G., Bezsonov, A.A., 2018. “Study of convergence of one-step adaptive identification algorithms”. Journal of Automation and Information Sciences, 50 (10), pp. 60-76.
   https://doi.org/10.1615/JAutomatInfScien.v50.i10.50
4. Rudenko, O.G., Bezsonov, O.O., “The Regularized Adaline Learning Algorithm for the Problem of Evaluation of Non-Stationary Parameters”, Control Systems and Computers, 2019, 1, pp.22-30. (In Russian).
   https://doi.org/10.15407/usim.2019.01.022
5. Huber, P., 1984. Robustness in statistics. M.: Mir, 304 p.
6. Hampel, F.R., Ronchetti, E.M., Rousseeuw, P.J., Stahel, W.A., Robust Statistics. The Approach Based on Influence Functions. N.Y.: John Wiley and Sons, 1986, 526 p.
7. Rudenko, O.G., Bezsonov, A.A., 2010.”Robust learning wavelet neural networks”, Journal of Automation and Information Sciences, 42(10), pp. 1-15.
   https://doi.org/10.1615/JAutomatInfScien.v42.i10.10
8. Rudenko, O., Bezsonov, O., 2011. “Function approximation using robust radial basis function networks”, J. of Intelligent Learning Systems and Applications, 3, pp.17-25.
   https://doi.org/10.4236/jilsa.2011.31003
9. Rudenko, O.G., Bezsonov, A.A., 2012. “Radial basic networks m-training by asymmetric influence functions”.Journal of Automation and Information Sciences, 44(2), pp. 48-64.
   https://doi.org/10.1615/JAutomatInfScien.v44.i2.50
10. Walach, E. Widrow, D., 1984. “The least mean fourth (LMF) adaptive algorithm and it’s family”, IEEE Trans, IT 30, pp. 275-283.
    https://doi.org/10.1109/TIT.1984.1056886
11. Chambers, J., Avlonitis, A., 1997. “A Robust Mixed-Norm Adaptive Filter Algorithm”, IEEE Signal Processing Letters, 4 (2), pp. 46-48.
    https://doi.org/10.1109/97.554469
12. Papoulis, V., Stathaki, T., 2004. “A Normalized Robust Mixed-Norm Adaptive Algorithm for System Identification”, IEEE Signal Processing Letters, 11 (1), pp. 56-59
    https://doi.org/10.1109/LSP.2003.819353
13. Chambers, J., Tanrikulu, O, Constantinides, G., 1984. Least mean mixed-norm adaptive filtering, Electronics Letters, 30 (19), pp. 1574-1575.
    https://doi.org/10.1049/el:19941060
14. Zerguine, A., 2012. A variable-parameter normalized mixed-norm (VPNMN) adaptive algorithm, EURASIP Journal on Advances in Signal Processing, 55, 13 p.
    https://doi.org/10.1186/1687-6180-2012-55
15. Rudenko, O.G., Bezsonov, O.O., Serdyuk, N.M. Olyynik, K.O., Romanyuk, O.S., 2019. “Robastna identyfikatsiya obʺyektiv za nayavnostyu nehausivsʹkykh zavad”, Byonyka yntellekta, Bionics of Intellect, 2(93), pp. 7-12. (In Ukrainian).
16. Rudenko, O.G., Bezsonov, O.O., Serdyuk, N.M. Olyynik, K.O., Romanyuk, O.S., 2020. “Robastna identyfikatsiya obʺyektiv na osnovi minimizatsiyi kombinovanoho funktsionalu”, Systemy obrobky informatsiyi, 2020,1 (160), pp.80-88. (In Ukrainian).
17. Principe, C., Xu, D., Zhao, Q., Fisher, J. W., 2000. “Learning from examples with information theoretic criteria”, J. VLSI Signal Process. Syst., 26 (1-2), pp. 61-77.
    https://doi.org/10.1023/A:1008143417156
18. Principe, C., Xu, D., Fisher, J., 2000. ” Information-theoretic learning”. Unsupervised Adaptive Filtering. New York: Wiley, pp. 265-319.
19. Chen, B., Hu, J., Pu, L., Sun, Z., 2007. “Stochastic gradient algorithm under (h, φ)-entropy criterion”, Circuits Syst. Signal Process, 26, pp. 941-960.
    https://doi.org/10.1007/s00034-007-9004-9
20. Santamaria, I., Pokharel, P.P, Jose, C., Principe, J.C., 2006. “Generalized Correlation Function: Definition, Properties, and Application to Blind Equalization”, IEEE Trans. on Signal Processing, 54, 6, pp. 2187-2197.
    https://doi.org/10.1109/TSP.2006.872524
21. Liu, W., Pokharel, P.P, Principe, J.C., 2007. “Correntropy: Properties and Applications in Non-Gaussian Signal Processing”, IEEE Trans. on Signal Processing, 1, pp. 5286-5298.
    https://doi.org/10.1109/TSP.2007.896065
22. Wang, W., Zhao, J., H. Qu, H., Chen, B., Principe, J.C., 2015. “An adaptive kernel width update method of correntropy for channel estimation”, IEEE International Conference on Digital Signal Processing (DSP), pp. 916-920.
    https://doi.org/10.1109/ICDSP.2015.7252010
23. Chen, B., Xing, L., Liang, J., Zheng, N, Principe, J.C., 2014. “Steady-state mean-square error analysis for adaptive filtering under the maximum correntropy criterion”, Signal Process. Lett. IEEE, 21(7), pp. 880-884.
    https://doi.org/10.1109/LSP.2014.2319308
24. Ma, W., Qua, H., Guib, G., Li, Xu L., Zhao, J., Chen, B., 2015. “Maximum correntropy criterion based sparse adaptive filtering algorithms for robust channel estimation under non-Gaussian environments”, J. of the Franklin Institute, 352 (2), pp. 2708-2727.
    https://doi.org/10.1016/j.jfranklin.2015.03.039
25. Guo, Y., Ma, B, Li, Y., 2016. “Kernel-Width Adaption Diffusion Maximum Correntropy Algorithm”, IEEE Acces, 4, pp.1-14.
26. Lu, L., Zhao, H., 2017. “Active impulsive noise control using maximum correntropy with adaptive kernel size”, Mechanical Systems and Signal Processing, 87, Part A, pp. 180-191.
    https://doi.org/10.1016/j.ymssp.2016.10.020
27. Qi, Y., Wang, Y., Zhang, J., Zhu, J., Zheng, X., 2014. Robust Deep Network with Maximum Correntropy Criterion for Seizure Detection, BioMed Research International, Article ID 703816, 10 p.
    https://doi.org/10.1155/2014/703816
28. Huang, F., Zhang, J., Zhang, S., 2017. “Adaptive filtering under a variable kernel width maximum correntropy criterion”, IEEE Trans. on Circuits and Systems II: Express Briefs, 64 (10), pp. 1247-1251.
    https://doi.org/10.1109/TCSII.2017.2671339
29. Perelman, I., 1982. Operativnaya identifikatsiya ob”yektov upravleniya. M.: Energoizdat, 272 p. (In Russian).
30. Rudenko, O.G., Terenkovskiy, I.D., Shtefan, A., Oda, G.A., 1998. “Modifitsirovannyy algoritm tekushchego regressionnogo analiza v zadachakh identifikatsii i prognozirovaniya”, Radioelektronika i informatika, 4(05), pp.58-61. (In Russian).
31. Goodwin, G., Sin, K.S., 2014. Adaptive filtering prediction and control. N.Y.: Dover Publications, 560 p.
32. Streng, G., 1980. Lineynaya algebra i yeye primeneniya. M.: Mir, 1980, 454 p.
33. Karchevskiy, Ye.M., Karchevskiy, M.M., 2018. Lektsii po lineynoy algebre i analiticheskoy geometrii: uchebnoye posobiye. Kazan’: Izd-vo Kazanskogo un-ta, 426 p. (In Russian).
34. Doyle, C., Stein, G., 1981. “Multivariable Feedback Design: Concepts for a Classical/Modern Synthesis”, IEEE Trans. on Automatic Control, 1981, 26, 1, pp. 4-16.
    https://doi.org/10.1109/TAC.1981.1102555
35. Zhao, S., B. Chen, B., Principe, J.C., 2012. “An adaptive kernel width update for correntropy”, Proc. of the International Joint Conference on Neural Networks (IJCNN ’12), Brisbane, Australia, pp. 1-5.
    https://doi.org/10.1109/IJCNN.2012.6252495
36. Jones, C., Marron, J.S., Sheather, S.J., 1996. “A brief survey of bandwidth selection for density estimation”, Journal of the American Statistical Association, 91, 433, pp. 401-407.
    https://doi.org/10.1080/01621459.1996.10476701
37. Silverman, W., 1986. Density Estimation for Statistics and Data Analysis, vol. 3: CRC Press: New York, NY, USA, 176 p.
    https://doi.org/10.1007/978-1-4899-3324-9
38. Bowman, W., 1984. “An alternative method of cross-validation for the smoothing of density estimates”, Biometrika, 71 (2), pp. 353-360.
    https://doi.org/10.1093/biomet/71.2.353
39. Scot, W., Terrell, G.R., 1987. “Biased and unbiased cross-validation in density estimation”, Journal of the American Statistical Association, 82, 400, pp. 1131-1146.
    https://doi.org/10.1080/01621459.1987.10478550
40. Shi, L., Zhao, H., Zakharov, Y., 2018. An Improved Variable Kernel Width for Maximum Correntropy Criterion Algorithm, IEEE Trans. on Circuits and Systems II: Express Briefs, 5p.

Received 14.06.2020.