Control Systems and Computers

https://doi.org/10.15407/csc.2023.03.033

Control Systems and Computers, 2023, Issue 3 (303), pp. 33-53.

UDK  004.65:004.75:004.738.5

O.A. Oursatyev, Ph.D. Eng. Science, Senior Research Associate, International Research and Training Centre for Information Technologies and Systems of the NAS and MES of Ukraine, ORCID: https://orcid.org/ 0009-0009-8323-0525, Acad. Glushkov ave., 40, Kiev, 03187, Ukraine, aleksei@irtc.org.ua

DATA RESEARCH IN INDUSTRIAL DATA MINING PROJECTS OF THE BIG DATA GENERATION ERA

Introduction. The review material is based mainly on business intelligence (BI) solutions designed for tasks with corporate data. But all the main aspects of working with data discussed in the work are also used on data processing platforms (Data Science Platform). Many BI vendors have expanded the capabilities of their systems to perform more advanced analytics, including Data Science. They added the phrase “Data Science” to their marketing research, and the term “advanced analytics” lost some popularity in relation to corporate data. The Data Science Platform provides a comprehensive set of tools for use by advanced users who traditionally work with data.

Capabilities that allow you to connect to multi-structured data across different types of storage platforms, both on-premises and in the cloud, and the infrastructure architecture of a modern BI analytics platform enable high-performance workloads, including business intelligence. It uses distributed architecture, massively parallel processing, data virtualization, in-memory computing, etc. The combination of traditional relational data processing with calculations on the well-known Apache Hadoop software infrastructure, which integrates a number of components of the Hadoop ecosystem (Apache Hive, HBase, Spark, Solr, etc.) with the necessary target functions, allows you to create a fully functional platform for storing and processing structured and non-structures data.

Purpose. A review of data processing problems and an analysis of the use of world-class mathematical apparatus and tools for obtaining knowledge from information were carried out.

Methods. The paper describes the use of Data Mining methods in big data processing tasks, as well as methods of business, recommendation and predictive analytics.

Result. The study suggests that machine learning-enhanced master data management (MDM), data quality, data preparation, and data catalogs will converge into a single, modern Enterprise Information Management (EIM) platform applicable to most new analytics projects. The results of the analysis of the process of identifying useful data can be useful to researchers and developers of modern platforms for processing and researching data in various spheres of society.

Conclusion. A review of data processing problems and an analysis of the use of world-class mathematical apparatus and tools for obtaining knowledge from information were carried out. It is shown that a high-quality solution to the problems of working with first-level data indicated in this review will be provided by data research in modern analytical platforms. Successful penetration into their essence at the level of obtaining knowledge using machine learning and artificial intelligence algorithms will make it possible to predict future results in managed objects (processes) and make informed decisions.

Download full text! (On Ukrainian)

Keywords: analytics maturity levels, predictive analytics, data discovery, data science, data preparation, self-service data, self-service data preparation platforms, data preparation conveyor.

1. Gritsenko, V.I., Oursatyev, A.A., 2017. “Big Data and Tools for Analytics”. Upravlausie sistemy i masiny, 4, pp. 3-14. (In Russian).
   https://doi.org/10.15407/usim.2017.04.003
2. Ginsburg, J., Mohebbi, M., Patel, R. et al. Detecting influenza epidemics using search engine query data. Nature. 2009, 457, pp. 1012-1014, http://www.nature.com/nature/journal/ v457/n7232/full/nature07634.html.
   https://doi.org/10.1038/nature07634
3. Gritsenko, V.I., Oursatyev, A.A., 2011. “Information Technologies: the Tendency, the Ways of the Development”. Upravlausie sistemy i masiny, 5, pp. 3-20. (In Russian).
4. Luhn, H.P., 1958. “A Business Intelligence System”. IBM Journal of Research and Development, Vol. 2, Issue 4, pp. 314-319.
   https://doi.org/10.1147/rd.24.0314
5. Martens, Ch. The maturing of BI. Interview: Hyperion chief strategy officer Howard Dresner discusses how as BI matures, companies should too. InfoWorld. Sep. 22. 2006. [online]. Available at: <https://www.infoworld.com/article/2661157/database/the-maturing-of-bi.html> [Accessed: 17 Dec. 2021].
6. Laurent Duval. Original Meaning of “Intelligence” in “Business Intelligence”. [online]. Available at: <https://datascience.stackexchange.com/questions/8016/original-meaning-of-intelligence-in-business-intelligence> [Accessed: 07 Nov. 2015].
7. What’s the Difference Between Business Intelligence (BI) and EPM? [online]. Available at: < http://blog.hostanalytics.com/whats-the-difference-between-business-intelligence-bi-and-epm> [Accessed: 07 Nov. 2021].
8. Dresner, H. Predicts the future of business intelligence. [online]. Available at: <http://searchbusinessanalytics.techtarget.com/podcast/Howard-Dresner-predicts-the-future-of-business-intelligence> [Accessed: 17 Dec. 2021].
9. Schlegel, K., Sallam, R.L., Yuen, D. et al. Magic Quadrant for Business Intelligence and Analytics Platforms. [online]. Available at: < http://business-view.dk/wp-content/uploads/2015/02/Magic-Quadrant-for-Business-Intelligence-and-Analytics-Platforms-ALL.pdf> [Accessed: 05 Feb. 2013].
10. Herschel, G., Linden, A., Kart, L. Magic Quadrant for Advanced Analytics Platforms. [online]. Available at: <https://pdfs.semanticscholar.org /1a9f /ff52e8084d0da00491e54d45113bd81d2e91.pdf> [Accessed: 19 Feb. 2014].
11. Herschel, G., Linden, A., Kart, L. Magic Quadrant for Advanced Analytics Platforms. [online]. Available at: <https://davidhoglund.typepad.com/files/magic-quadrant-for-advanced-analytics-platforms.pdf> [Accessed: 17 Feb. 2015].
12. Magic Quadrant for Business Intelligence and Analytics Platforms / R.L. Sallam, J. Tapadinhas, J. Parenteau et al. [online]. Available at: <http://www.thgcfo.com/wp-content/uploads/2014/02/Magic-Quadrant-for-Business-Intelligence-and-Analytics-Platforms.pdf> [Accessed: 20 Feb. 2014].
13. Magic Quadrant for Business Intelligence and Analytics Platforms / Rita L. Sallam, Bill Hostmann, Kurt Schlegel et al. [online]. Available at: <http://zzircon.com/wp-content/uploads/2015/04/Magic-Quadrant-for-Business-Intelligence-and-Analytics-Platforms-2015.pdf> [Accessed: 23 Feb. 2015].
14. Business intelligence systems 2013. Market overview. TAdviser. [online]. Available at: <http://www.old.rbcgrp.com/files/QlikView_TAdviser2013.pdf> [Accessed: 17 Dec. 2021].
15. Dinsmore, T.W. Disruptive Analytics: Charting Your Strategy for Next-Generation Business Analytics. Apress, 2016, 262 p, https://www.apress.com/us/book/9781484213124> [Accessed: 17 Dec. 2021].
    https://doi.org/10.1007/978-1-4842-1311-7
16. Nikolaev, O. Gartner: analytics should become a top priority for business. [online]. Available at: <http://channel4it.com/publications/Gartner-analitika-dolzhna-stat-glavnym-prioritetom-dlya-biznesa-5204.html#> [Accessed: 17 Oct. 2014].
17. Chabrier, A. Data Types for Data Sciences. [online]. Available at: <https://towardsdatascience.com/data-types-for-data-sciences-65dcbda6177c1818> [Accessed: 17 Dec. 2021].
18. Prohnozna analityka vid SAP – SAP. Predictive Analytics. [online]. Available at: <https://jetbi.ru/obzor-sap-predictive-analytics> [Accessed: 23 Aug. 2018].
19. Predictive analytics. [online] Available at:<https://en.wikipedia.org/wiki/Predictive_ analytics> [Accessed 23 Aug. 2018].
20. Stepashko, V.S., Yefimenko, S.N, 2018. “Predictive Analytics as an effective tool for decision support in Digital Economics Systems”. Upravlausie sistemy i masiny, 6, pp. 25-35. (In Ukraine).
    https://doi.org/10.15407/usim.2018.06.025
21. Stepashko, V.S., 2017. “The Achievements and Prospects of Inductive Modeling”. Upravlausie sistemy i masiny, 2, pp. 58-73. (In Russian).
    https://doi.org/10.15407/usim.2017.02.058
22. Descriptive, Predictive, and Prescriptive Analytics Explained. [online] Available at:<https://halobi.com/blog/descriptive-predictive-and-prescriptive-analytics-explained/> [Accessed 05 Jun. 2019].
23. Prescriptive Analytics. [online] Available at:<https://en.wikipedia.org/ wiki/Prescriptive_analytics>[Accessed 07 May 2019].
24. Predictive Analytics vs. Prescriptive Analytics: What Is the Difference? Available at:<https://www.proponent.com/predictive-analytics-vs-prescriptive-analytics/> [Accessed 07 May 2019].
25. Descriptive, Predictive and Prescriptive Analytics. Available at:<http://www.gurobi.com/resources/prescriptive-analytics>[Accessed 07 May 2019].
26. IBM Analytics. Prescriptive analytics. [online] Available at:<https://www.ibm.com/ analytics/prescriptive-analytics> [Accessed 07 May 2019].
27. Frankenfield, J., Prescriptive Analytics. [online] Available at: <https://www.investopedia.com/terms/p/prescriptive-analytics.asp> [Accessed 06 March 2019].
28. Schlegel, K., 2008. The Rise of Data Discovery Tools. [online] Available at:<https://www.gartner.com/en/ documents/765514/the-rise-of-data-discovery-tools> [Accessed: 17 Dec. 2021].
29. A Closer Look at One of 2017’s Most Important BI Trends. [online] Available at:<https://bi-survey.com/data-discovery> [Accessed: 17 Dec. 2021].
30. Magic Quadrant for Data Science and Machine-Learning Platforms / Peter Krensky, Erick Brethenoux, Carlie Idoine et al. [online] Available at:<https://www.gartner.com/ en/documents/3860063> [Accessed: 22 Feb. 2018].
31. Magic Quadrant for Data Science and Machine Learning Platforms. Carlie Idoine, Peter Krensky, Alexander Linden et al. [online] Available at:<https://www.gartner.com/en/documents/ 3899464/magic-quadrant-for-data-science-and-machine-learning-pla> [Accessed: 28 Jan. 2019].
32. Elliott, T., 2017. What is Artificial Intelligence Called?! [online] Available at: <https:// timoelliott.com/blog/2017/06/what-is-artificial-intelligence-called.html> [Accessed 11 Jul. 2019].
33. Sallam, R.L., Tapadinhas, J., Parenteau J. and et al. Magic Quadrant for Business Intelligence and Analytics Platforms. [online] Available at:<http://www.thgcfo.com/wp-content/uploads/2014/02/Magic-Quadrant-for-Business-Intelligence-and-Analytics-Platforms.pdf> [Accessed: 17 Feb. 2014].
34. Oursatyev, A.A., 2017. “Big data. Analytical databases and data warehouse: Teradata”, Upravlausie sistemy i masiny, 2, pp. 51-67. (In Russian).
    https://doi.org/10.15407/usim.2018.02.051
35. Harris, J. Five stages of data preparation. [online] Available at:<https://www.sas.com/ru_ua/insights/articles/ data-management/the-five-d-s-of-data-preparation.html> [Accessed: 17 Dec. 2021].
36. Ehtisham, Z., Sallam, R.L., Shubhangi, V. Market Guide for Data Data Preparation. [online] Available at:<https://www.gartner.com/doc/reprints?id=1-4FSMSCI&ct=170929&st=sb> [Accessed: 17 Dec. 2017].
37. Howard, Ph. Data Preparation (self-service). [online] Available at:<https://www.bloorresearch.com/technology/data-preparation-self-service/> [Accessed: 17 Dec. 2021].
38. BI Trends: Table of Contents. [online] Available at:<https://bi-survey.com/self-service-bi> [Accessed: 17 Dec. 2021].
39. The Definitive Guide to Self-Service Data. https://resources.boomi.com/resources/ home/the-definitive-guide-to-self-service-data> [Accessed: 17 Dec. 2021].
40. Oursatyev A.A., 2019. “Big data. Analytical databases and data warehouse: Greenplum”, Upravlausie sistemy i masiny, 2, pp. 40-69. (In Russian).
    https://doi.org/10.15407/usim.2019.02.040
41. Patel, M. Chorus Brings Data Science Minds Together. Feb., 2013. [online]. Available at: <https://blog.dellemc.com/ en-us/chorus_data_science/> [Accessed: 17 Dec. 2021].
42. Greenplum Software Introduces Greenplum Chorus. Originally published April 12 2010. [online]. Available at: <http://www.b-eye-network.com/view/13182> [Accessed: 17 Dec. 2021].
43. Howard, Ph. Self-service data preparation and cataloguing. [online]. Available at: <https://www.bloorresearch.com/research/self-service-data-preparation-cataloguing-p2/> [Accessed: 12 Nov. 2016].
44. Howard, Ph. Data Preparation (self-service). [online]. Available at: <https: //www.bloorresearch.com/technology/data-preparation-self-service/> [Accessed: 01 July 2018].
45. Oursatyev, A.A., 2018. “Big data. Analytical databases and data warehouse: Vertica, Kdb”. Upravlausie sistemy i masiny, 1, pp. 57-70. (In Russian).
    https://doi.org/10.15407/usim.2018.01.057
46. Data profiling. [online]. Available at: <https://en.wikipedia.org/wiki/Data_profiling> [Accessed: 17 Dec. 2021].
47. Oursatyev, A.A., 2016. “SomeFrameworks forAnalytics Big Data”. Upravlausie sistemy i masiny, 3, pp. 29-42. (In Russian).
    https://doi.org/10.15407/usim.2016.03.029
48. Estensen, F.O. Master Data Management BI MicrosoftMDM. [online]. Available at: <https://ru.scribd.com/ presentation/252578258/BI-MicrosoftMDM-Frank-Olav-Estensen#scribd> [Accessed: 17 Dec. 2021].
49. Garter Bimoda. [online]. Available at: <https://www.gartner.com/en/information-technology/glossary/bimodal> [Accessed: 17 Dec. 2021].
50. Zeichick, A. Mode 1, Mode 2: Alan Zeichick on Bimodal Development. [online]. Available at: <https://blog.parasoft.com/mode-1-mode-2-alan-zeichick-on-bimodal-development> [Accessed: 17 Dec. 2015].

Received 06.07.2022