Control Systems and Computers

https://doi.org/10.15407/csc.2022.03.039

Control Systems and Computers, 2022, Issue 3 (299), pp. 39-52

UDK 004.383.3

A.K. Sieriebriakov, Leading Engineer, International Research and Training Center for Information Technologies and Systems NAS and MES of Ukraine, Glushkovave., 40, Kyiv, 03187, Ukraine,sier.artem1002@outlook.com

Yu.P. Bogachuk, Ph.D. (Engineering), Senior Researcher, Leading Researcher, International Research and Training Center for Information Technologies and Systems of the National Academy of Sciences of Ukraine and Ministry of Education and Science of Ukraine, Acad. Glushkova ave., 40, Kyiv, 03187, Ukraine, bip47@ukr.net

S.O. Bondar, International Research and Training Center for Information Technologies and Systems of the National Academy of Sciences of Ukraine and Ministry of Education and Science of Ukraine, Acad. Glushkova ave., 40, Kyiv, 03187, Ukraine, seriybrm@gmail.com,

V.M. Simakhin, International Research and Training Center for Information Technologies and Systems of the National Academy of Sciences of Ukraine and Ministry of Education and Science of Ukraine, Acad. Glushkova ave., 40, Kyiv, 03187, Ukraine, Thevladsima@gmail.com

FUNDAMENTAL FREQUENCIES CONTOUR EXTRACTION BASED ON THE EXTENDED HARMONIC-PERCUSSIVE SOURCE SEPARATION

Amidst the multiplicity of audio signals processing tasks, the problem of harmonic features extraction occupies special place. In the case of analyzing a musical composition, the task is reduced to extracting a melodic contour from such sound source. To solve it, an attempt was made to combine the method of median filtering with the salience estimation method, for their application at various stages of the analysis of the input audio signal. A combination of approaches is used to obtain -representation of the melody, based on the processing of the filtered values ​​obtained in the first step.

The purpose of the article is to obtain the trajectory of -values ​​of the input composition and filter the harmonics corresponding to this trajectory, which together represent the melody. The proposed method is promising for use in audio signal processing systems for melody and harmonic contour extraction and its further reuse.

The spectrum decomposition, based on the tendency of the distribution of its composing sounds, allows to effectively select melodic contours from non-melodic contours. Despite this, there is a need for further research regarding the distribution model of harmonic-percussion characteristics relative to each other. Such a model should be extended with heuristics for more accurate filtering of different melodic lines of polyphonic music.

Download full text! (On English)

Keywords: melody, harmonic-percussive source separation, fundamental frequency trajectory, audio signal, median filtering, salience function, musical tone, note, fundamental tone, harmonic, melodic line, melodic contour, windowed Fourier transform, logarithmic frequency scale, harmonic summation.

1. Sargent, G., Bimbot, F., & Vincent, E. (2016). “Estimating the Structural Segmentation of Popular Music Pieces Under Regularity Constraints”. IEEE/ACM Transactions on Audio, Speech, and Language Processing. pp. 344 -358. 
   https://doi.org/10.1109/TASLP.2016.2635031
2. Canadas-Quesada, F. J., Vera-Candeas, P., Ruiz-Reyes, N., Carabias-Orti, J., & Cabanas-Molero, P. (2014). “Percussive/harmonic sound separation by non-negative matrix factoriza-tion with smoothness/sparseness constraints”. EURASIP Journal on Audio, Speech, and Mu-sic Processing. 
   https://doi.org/10.1186/s13636-014-0026-5
3. Bosch, J. J., Marxer, R., & Gómez, E. (2016). “Evaluation and combination of pitch estimation methods for melody extraction in symphonic classical music”, Journal of New Music Research, 45 (2), pp. 101-117, DOI: 
   https://doi.org/10.1080/09298215.2016.1182191
4. Salamon, E., Gomez, D.P., Ellis, W., Richard, G., 2014. “Melody Extraction from Polyphonic Music Signals: Approaches, applications, and challenges,” IEEE Signal Processing Magazine, vol. 31, no. 2, pp. 118-134, March 2014.
   https://doi.org/10.1109/MSP.2013.2271648
5. Raczyński, S. A., Ono, N., & Sagayama, S.  (2007). “Multipitch Analysis with Harmonic Nonnegative Matrix Approximation”.  8th International Conference on Music Information Retrieval, pp. 381-386.
6. Cella, C.-E. (2010). “Harmonic Components Extraction in Recorded Piano Tones”. 128th Audio Engineering Society Convention. 2010, May 22–25 London, UK. http://www.aes.org/e-lib/browse.cfm?elib=15408.
7. Prince, J. (2014). “Contributions of Pitch Contour, Tonality, Rhythm, and Meter to Melodic Similarity”. Journal of Experimental Psychology Human Perception & Performance. In press. 
   https://doi.org/10.1037/a0038010
8. Salamon, J., Peeters, G., & Röbel, A. (2012). “Statistical characterisation of melodic pitch contours and its application for melody extraction”. Proceedings – 13th International Society for Music Information Retrieval Conference, pp. 187-192.
9. Canadas-Quesada, F. J., Vera-Candeas, P., Ruiz-Reyes, N., Munoz-Montoro, A., & Bris-Penalver, F. J.  (2016). “A method to separate musical percussive sounds using chroma spectral flatness”. SIGNAL, 51.
10. Fitzgerald, D., Liutkus, A., Rafii, Z., Pardo, B., Daudet, L., (2014). “Harmonic/Percussive Separation Using Kernel Additive Modelling”. In 25th IET Irish Signals & Systems Confer-ence 2014 and 2014 China-Ireland International Conference on Information and Communi-cations Technologies, pp. 35-40. DOI: 
    https://doi.org/10.1049/cp.2014.0655
11. Fitzgerald, D., (2010). “Harmonic/Percussive Separation using Median Filtering”. 13th International Conference on Digital Audio Effects (DAFx-10).
12. Driedger, J., Müller, M., Disch, S., (2014). “Extending Harmonic-Percussive Separation of Audio Signals”. In ISMIR, pp. 611-616.
13. Füg, R., Niedermeier, A., Driedger, J., Disch, S., & Müller, M. (2016). “Harmonic-percussive-residual sound separation using the structure tensor on spectrograms,” IEEE In-ternational Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 445-449.
    https://doi.org/10.1109/ICASSP.2016.7471714
14. Ono, N., Miyamoto, K., Le Roux, J., Kameoka, H., & Sagayama, S. (2008). “Separation of a monaural audio signal into harmonic/percussive components by complementary diffusion on spectrogram”. In 2008 16th European Signal Processing Conference, IEEE., pp. 1-4.
15. Degani, A., Leonardi, R., Migliorati, P., Peeters, G. (2014). A Pitch Salience Function Derived from Harmonic Frequency Deviations for Polyphonic Music Analysis. In DAFx, pp. 195-201. 13140/2.1.4409.7922.
16. Zhang, W., Chen, Z., Yin, F. (2018). “Melody Extraction Using Chroma-Level Note Track-ing and Pitch Mapping”. Applied Sciences. 8 (9), 1618.
    https://doi.org/10.3390/app8091618

Received 01.10.2022