Control Systems and Computers

https://doi.org/10.15407/usim.2019.01.022

Upr. sist. maš., 2019, Issue 1 (279), pp. 22-30.

UDC 004.852

O.G. RUDENKO, Doctor (Eng.), Professor, Head of Information Systems Departament at Simon Kuznets Kharkiv National University of Economics, Nauky ave., 9-A, Kharkiv, 61166, Ukraine, oleg.rudenko@hneu.net

O.O. BEZSONOV, Doctor of Technical Sciences, Assosciate Professor, Professor of Information Systems Departament at Simon Kuznets Kharkiv National University of Economics, Nauky ave., 9-A, Kharkiv, 61166, Ukraine, oleksandr.bezsonov@hneu.net

THE REGULARIZED ADALINE LEARNING ALGORITHM FOR THE PROBLEM OF EVALUATION OF NON-STATIONARY PARAMETERS

Introduction. Among the most simple in terms of calculation of one-step algorithms for identification, the most effective are Kachmazh and Nagumo-Noda algorithms. However, their speed is often not sufficient in the evaluation of non-stationary parameters. Knowledge (or approximation) of the drift law allows us to obtain an effective algorithms for tracking non-stationary parameters. It should, however, be noted that the errors in the task of the parameters change can lead to the properties loss of the algorithm convergence. The lack of information about the nature of drift requires the development of identification algorithms that use the minimum amount of information and maintain efficiency over a wide range of variation of parameters. Therefore, it is natural to study the dynamic properties of specific algorithms and determine the maximum achievable accuracy of tracking, which will ultimately determine the most effective areas to use the algorithms and develop the recommendations for their practical application.

Purpose is to study the properties of a modified regularized Kachmazh algorithm, develop recommendations for its practical application.

Methods. Research methods are based on the theory of identification. On its basis, the properties of the modified regularized Kachmazh algorithm are investigated. Also, methods of simulation is used, which allow to confirm the effectiveness of the obtained results and to develop the recommendations for their practical use.

Results. The conditions of the regularized Kachmazh algorithm convergence are determined in the estimation of nonstationary parameters in the presence of noise disturbances. Non-asymptotic and asymptotic estimates are rather general and depend on both the degree of non-stationary object and the statistical characteristics of the noise.

Conclusion. As the results of the research have shown, the use of the regularization application in identification algorithms improves the stability of the algorithms, but leads to some slowdown in the process of the model constructing. The convergence conditions of the regularized Kachmazh algorithm are determined in the estimation of the nonstationary parameters in the presence of the noise disturbances. Non-asymptotic and asymptotic estimates are rather general and depend on both the degree of non-stationary object and the statistical characteristics of the noise. If these parameters are not known, then it is necessary to use some recursive procedure for their evaluation and use the obtained estimates to clarify the parameters included in the algorithms.

 Download full text! (In Russian)

Keywords: regularized algorithm, convergence, asymptotic estimates, recurrent procedure, law of drift.

1. Dupac, V., 1965. “A dynamic stochastic approximation method”. Ann. Math. Statist., 36, pp. 1695-1702.
   https://doi.org/10.1214/aoms/1177699797
2. Tsypkin, Ya.Z., Kaplinskiy, A.I., Larionov, K.A., 1970. “Algoritmy adaptatsii i obucheniya v nestatsionarnykh usloviyakh”, Izv. AN SSSR. Tekhn. Kibernetika, 5, pp. 9-2. (In Russian).
3. Kaczmarz, S., 1937. Angenäherle Auflösung von Systemen linearer Gleichungen. Bull. Int. Acad. Polon. Sci. Lett., C 1, Sci. Math. Nat., Ser. A, pp. 355-357.
4. English translation: Kaczmarz S., 1993. Approximate solution of systems of linear equations, Int. J. of Control, 57, pp. 1269-1271.
   https://doi.org/10.1080/00207179308934446
5. Nagumo, I., Noda, A., 1967. “A learning method for system identification”, IEEE Trans. Autom. Control, AC-12, 3, pp. 282-287.
   https://doi.org/10.1109/TAC.1967.1098599
6. Okrug, A.I., 1981. “Dinamicheskiy algoritm Kachmazha”, Avtomatika i telemekhanika, 1981, 1, pp. 74-79. (In Russian).
7. Widrow, B., Hoff, M., 1960. “Adaptive switching circuits”, IRE WESCON Convention Record. Part 4. New York: Institute of Radio Engineers, pp. 96-104.
   https://doi.org/10.21236/AD0241531
8. Widrow, B., Lehr, M., 1990. “30 years of adaptive neural networks: perceptron, madaline, and backpropagation”, Proc. IEE, 78 (9), pp. 1415-1442.
   https://doi.org/10.1109/5.58323
9. Chadeyev, V.M., 1964. “Opredeleniye dinamicheskikh kharakteristik obíektov v protsesse ikh normal’noy ekspluatatsii dlya tseley samonastroyki”, Avtomatika i telemekhanika, 25 (9), pp. 1302-1306. (In Russian).
10. Raybman, N.S., Chadeyev, V.M., 1966. Adaptivnyye modeli v sistemakh upravleniya, M.: Sov. Radio, 156 p. (In Russian
11. Rudenko, O.G., 1982. “Otsenka skorosti skhodimosti odnoshagovykh ustoychivykh algoritmov identifikatsii”. Doklady AN USSR, Ser. A. Fiz-mat i tekhn. nauki, 1, pp.64-66. (In Russian).
12. Ciochina, S., Paleologu, C., Benesty, J., 2016. “An optimized NLMS algorithm for system identification”, Signal Processing, 2016, 118, pp. 115-121.
    https://doi.org/10.1016/j.sigpro.2015.06.016
13. Khong, A.W.H., Naylor, P.A., 2007. “Selective-tap adaptive filtering with performance analysis for identification of timevarying systems”, IEEE Trans. Audio, Speech, and Language Processing, 15, pp. 1681-1695.
    https://doi.org/10.1109/TASL.2007.896671
14. Benesty, J., Paleologu, C., Ciochinǎ, S., 2011. “On regularization in adaptive filtering”, IEEE Trans. Audio, Speech, Language Process, 19, pp. 1734-1742.
    https://doi.org/10.1109/TASL.2010.2097251
15. Paleologu, C., Ciochinǎ, S., Benesty, J., Grant, S., 2015. An overview on optimized NLMS algorithms for acoustic echo cancellation, EURASIP J. Adv. Sig. Proc., 97, 19 p.
    https://doi.org/10.1186/s13634-015-0283-1
16. Bershad, N.J., McLaughlin, S., Cowan, C.F.N., 1990. “Performance comparison of RLS and LMS algorithms for tracking a first order Markov communications channel”, Proc. IEEE Int. Symp. on Circuits and Syst., pp. 266-270.
17. Wagner, K., Doroslovacki, M., 2008. “Towards analytical convergence analysis of proportion-type NLMS algorithms”, Proc. IEEE Int. Conf. Acoustics Speech Signal Processing, pp.3825-3828.
    https://doi.org/10.1109/ICASSP.2008.4518487
18. Liberol’, B.D., Rudenko, O,G., Bessonov, A.A., 2018. “Issledovaniye skhodimosti odnoshagovykh adaptivnykh algoritmov identifikatsii”. Problemy upravleniya i informatiki., 5, p. 19-32. (In Russian).

Received 09.01.2019