Control Systems and Computers

DOI: https://doi.org/10.15407/usim.2017.04.003

Upr. sist. maš., 2017, Issue 4 (270), pp. 3-14.

UDC 004.65:004.7:004.75:004.738.5

Grіtsenko Volodymyr I., Corresponding member of the Ukrainian academy of sciences, Director, International Research and Training Center for Information Technologies and Systems of the NAS and MES of Ukraine, Glushkov ave., 40, Kyiv, 03187, Ukraine, E-mail: vig@irtc.org.ua

Oursatyev Alexey A., PhD in Techn. Sciences, Leading Research Associate, International Research and Training Centre of Information Technologies and Systems of the NAS and MES of Ukraine, Glushkov ave., 40, Kyiv, 03187, Ukraine, aleksei@irtc.org.ua

Big Data and Tools for Analytics

Introduction. The materials of the well-known foreign analytical corporations, the research IT-companies, and the international scientific centers are analyzed. The influence on transformation of the IT to a new set of key technologies creating the platform for their processing is traced through a prism of the properties and characteristics, the nature and potential value of Big Data.
Purpose. It is important to create and research the methods and technologies for processing the Big Data for extracting new knowledge, discovering non-obvious links and in-depth understanding of the phenomena and the investigated processes and the prospects of their development.
Methods. The informational and analytical methods and technologies for data processing, the methods for data assessment and forecasting, taking into account the development of the most important areas of the informatics and information technology.
Results. Consolidated analysis, even with a small number of external data sources, is too expensive. Actually, creating a
DB takes time and the significant resources for putting them into operation. In this regard, interest shifts from the traditional
stores to hybrid and logical solutions. In a limited application it is possible to use a flexible approach aimed at reducing the dimension of the Big Data.

The storages orientation for the widespread use in the intelligent analytical processing problems, with their inherent deep insight into the investigated phenomena, requires the simple access to all available types of the information resources, the possibility of using them in the different way, and the delivery of the required data to the application resources without their movement.

Agreeing on the concept of the logical storage in general, we imagine the LDW storage as a loosely coupled, providing the necessary flexibility, multi-layered architecture, which includes new technologies and the progressive means of identifying, extracting and preparing a wide range of data. In this case, the analysis, taking into account the logic of the information processing, provides a certain information resource to the application as a service. This will ensure the simple availability and relevance of the information contained in the Big Data.

Conclusion. Big Data is a complex inter-branch scientific and technical problem. In the future, the Big Data processing technologies, with the inclusion into its contours the methods and means of intellectualization, will be widely used in the economic, industrial and technological spheres and other important fields of human activity. 

Keywords: Big Data, multi-structured data, data management solution for analytics, analytic warehouse, logical data warehouse.

Download full text!

1. Big Data: The next frontier for innovation, competition, and productivity. J. Manyika, M. Chui, B. Brown et al., May 2011, http://www.mckinsey.com/businessfunctions/business-technology/our-insights/big-data-the-next-frontier-for-innovation
2. EMC. Research and analysis of IDC «Digital universe study» commissioned by EMC Corporation EMC, 2014, http://ukraine.emc.com/leadership/digital-universe/index.htm#Archive (In Russian).
3. The Digital Universe of Opportunities: Rich Data and the Increasing Value of the Internet of Things. EMC Digital Universe with Research & Analysis by IDC, April 2014, http://www.emc.com/leadership/digital-universe/2014iview/index.htm
4. The expanding digital universe March 2007. J.F. Gantz, D. Reinsel, Chute Chr. et al., March, 24, 2015,
5. Gantz J., Reinsel D. The Digital Universe Decade – Are You Ready?, May 2010, http://www.emc.com/collateral/analyst-reports/idc-digital-universe-are-you-ready.pdf
6. EMC NEWS. Press Rellease. New Digital Universe Study Reveals Big Data Gap: Less Than 1% of World’s Data is
7. Gantz, J., Reinsel, D. The Digital Universe in 2020: Big Data, Bigger Digital Shadows, and Biggest Growth in the Far East, Dec. 2012, http://www.emc.com/collateral/analyst-reports/idc-the-digital-universe-in-2020.pdf
8. Lesk M. How Much Information Is There In the World?, 1997, www.lesk.com/ mlesk/ksg97/ksg.html
9. Lyman P., Hal R. Varian How much information? 2003 (School of Information Management and Systems, Univ. of California at Berkeley). – http://www2.sims.berkeley.edu/research/projects/how-much-info-2003/
10. Hilbert M., López P. The World’s Technological Capacity to Store, Communicate, and Compute Information, Published Online Feb. 10 2011, Science 1 April 2011, 332, 6025, pp. 60–65. DOI: 10.1126/science. 1200970, http://www.sciencemag. org/content /332/6025/ 60.full
11. Big Data. Nature, 2008, 455, 7209, pp. 1–136, http://www.nature.com/nature/journal/v455/n7209/in-dex.html
12. Marx, V. Biology: The big challenges of big data. Nature International weekly journal of science, 2013, 498, N 7753, P. 255–260, http://www.nature.com/nature/journal/ v498/n7453/full/498255a.html.
13. Olofson Carl W., Vesset Dan Big Data: Trends, Strategies, and SAP Technology, August 2012, https://www.sap.
14. Gantz J., Reinsel D. Extracting Value from Chaos, June 2011, https://www.emc.com/collateral/analyst reports/idc-extracting-value-from-chaos-ar.pdf
15. Chui M., Loffler M., Roberts R. The Internet of Things. McKinsey Quarterly, March 2010, http://www.mckinsey.com/industries/high-tech/our-insights/the-in-ternet-of-things
16. Uont, R., Shilit, B., 2015. “The mechanisms of the Internet of things”, Otkrytye sistemy, 1, pp. 38– 42. (In Russian).
17. Oracle: Big Data for the Enterprise. June 2013, http://www.oracle.com/us/products/database/big-data-for-enterprise-519135.pdf
18. Gritsenko, V.I., Oursatyev, A.A., 2011. “Information Technologies: the Tendency, the Ways of the Development”. Upravlâûŝie sistemy i mašiny, 5, pp. 3–20. (In Russian).
19. Detecting influenza epidemics using search engine query data. J. Ginsburg, M. Mohebbi, R. Patel et al., Nature, 2009, 457, P. 1012–1014, http://www.nature.com/nature/journal/v457/n7232/full/nature07634.html
20. Asadullaev S. Data Warehouse Architectures-1, -2, -3, 2009, http://www.ibm.com/developerworks/ru/library/sabir/axd_1/index.html … axd_3/ index.html (In Russian).
21. Asadullaev S. Data, metadata and NSI: triple storage strategy. – 2009, http://www.ibm.com/developer-works/ru/library/r-nci/index.html (In Russian).
22. The Practice of Building Data Warehousing: The SAS System, Open systems, 1998, n 4–5, http://www.osp.ru/dbms/1998/04-05/13031592/ (In Russian).
23. Mark A. Beyer, Roxane Edjlali. Magic Quadrant for Data Warehouse and Data Management Solutions for Analytics, 12 Feb. 2015, http://www.gartner.com/technology/reprints.do?id=1-2A21OQO&ct=150217&st=sg
24. Roxane Edjlali, Mark A. Beyer. Magic Quadrant for Data Warehouse and Data Management Solutions for Analytics, 25 Feb. 2016, https://www.gartner.com/doc/reprints?id=1-2ZFVZ5B&ct=160225&st=sb.
25. Data warehouses: the market is being transformed. On the materials of foreign sites. Intersoft Lab, 2012,
26. TDWI. The Logical Data Warehouse: What it is and why you need it., June 24, 2015, https://tdwi.org/webcasts/2015/06/the-logical-data-warehouse-what-it-is-and-why-you-need-it.aspx
27. ThoughtWeb. Logical Data Warehousing for. Gartner , http://imagesrv. gart-ner.com/media-products/pdf/samples/ sample3.pdf .
28. Column DBMS – the principle of operation, advantages and scope. – 28 Jan. 2011, http://habrahabr.ru/post/95181/ (In Russian).
29. Whitehorn M. Big Data Technologies emerge to battle large, complex data sets, 05 Dec. 2011, http://www.computerweekly.com/news/2240111952/Big-data-tech-nologies-emerge-to-battle-large-complex-data-sets or https://www.prj-exp.ru/dwh/big_data_technologies_emer-ge_to_battle.php

Received 08.08.2017