Control Systems and Computers

https://doi.org/10.15407/csc.2022.02.003

Control Systems and Computers, 2022, Issue 2 (298), pp. 3-10

UDC 004.932

Krygin Valerii M. Ph.D. student, junior researcher,IRTC for IT and Systems of the NAS of MES of Ukraine, Glushkovave., 40, Kyiv, 03187, Ukraine, e-mail: valeriy.krygin@gmail.com

ORCID: https://orcid.org/0000-0002-9000-1685

Shylo Maksym K. master student,National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”,Peremogyave., 37, Kyiv,03056,Ukraine, e-mail: maksym.shylo.work@gmail.com

ORCID: https://orcid.org/0000-0001-5762-2546

Creating Slides from Video Lecture

Introduction. Video recordings of lectures are no longer a rarity in the conditions of distance learning. Videos may be in an inconvenient format for students or contain different artifacts due to compression, camera quality, and other factors. It is useful to have a presentation of the study material, which contains only the text from the board because such a view of the material is most like the compendium.

Purpose.The item of our work is creating an algorithm for obtaining panorama slides without a teacher from video lecture. To develop this algorithm, we use the Boykov-Kolmogorov Max-flow algorithm for obtaining a mask of moving objects. In some video lectures, the camera is moved so that the teacher is in sight, so some of the recordings are not visible. To do this, we make frames stitching to get a panorama. To reduce duplication of slides due to camera shaking, we perform video-stabilization as a preprocessing step, then replace the pixels using the mask. Finally, we create slides by comparing changes in the frames, binarizingand denoising.

Methods. We used the Boykov-Kolmogorov Max-flow algorithm, SIFT, Laplace operator and Otsu binarization for developing information technology.

Results.As a result, we get panorama slides with the extracted text or drawings from the board, which will help to simplify the creation of e-learning materials for both new and existing lecture recordings. Teachers will also be able to quickly provide lecture material to students even if they teach several complex subjects.

Conclusion. We have developed an algorithm and implemented information technology for obtaining slides from video-lecture. Now, a teachercan give students a short content of video material. The next steps are automatic detection of the board, division of the board into sections, and recognition of the text and formulas written on the board.

 Download full text! (On Ukrainian)

Keywords: intellectual video processing, images processing, video stabilization.

1. Patent Ukrayiny no H04N7/00, G02B27/00, G06T7/215. Sposib peretvorennya videozapysu z doshky u slayd-shou / V.M Kryhin. no u202100547; zayavl. 10.02.2021; opubl. 01.09.2021, Byul no 35. 4 p.
2. Zhang, Z., He, L.W., 2007. White board scanning and image enhancement. Digital Signal Pro-cessing, 17(2), pp. 414-432. [Online] Available at: <http://www.sciencedirect.com/science/article/pii/S1051200406000595> [Accessed 27 May. 2021]. https://doi.org/10.1016/j.dsp.2006.05.006
3. Gonzalez, A., Suh, B., Choi, E., 2012. Whiteboard disclosure using back ground subtraction and object tracking.
4. He, L.W., Zhang, Z., 2004. Real-time white board capture and processing using a video camera for teleconferencing. Tech. Rep. MSR-TR-2004-91, [Online] Available at: <https://www.microsoft.com/en-us/research/publication/real-time-whiteboard-capture-and-processing-using-a-video-camera-for-teleconferencing/> [Accessed 27 May. 2021].
5. Davila, K., Zanibbi, R., 2017. “White board video summarization via spatio-temporal conflict minimization”. 14th IAPR International Conference on Document Analysis and Recognition (ICDAR). vol. 01, pp. 355-362. 
   https://doi.org/10.1109/ICDAR.2017.66
6. Pavlyuk, A.D, Kryhin, V.M, Tkach, V.M., 2021. “Vidstezhennya periodychnoho rukhu obʺyektiv zi statychnoyi kamery na prykladi videoihor”. Teoretychni i prykladni problemy fizyky, matematyky ta informatyky. 2021. T 2. no 19. S. 135-137. [Online] Available at: <https://drive.google.com/file/d/1V9zQB2Jvgcaw3o7eEkzMmviLRrhrHbpD/view/> [Accessed 14 Dec. 2021].
7. Boykov, Y., Kolmogorov, V. 2004. “An experimental comparison of min-cut/max-flow algo-rithms for energy minimization in vision,” IEEE Trans actions on Pattern Analysis and Machine Intelligence, vol. 26, no 9, pp. 1124-1137, https://doi.org/10.1109/TPAMI.2004.60
8. Lucas, B. D., Kanade, T., 1981. “An iterative image registration technique with an application to stereo vision”. Vol. 81. Proceedings of the 7th international joint conference on Artificial intelligence – Volume 2 (IJCAI’81). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA pp. 674-679.
9. Farneback, G., 2003. “Two-Frame Motion Estimation Based on Polynomial Expansion”. Bigun J., Gustavsson T. (eds) Image Analysis. SCIA 2003. Lecture Notes in Computer Science, vol 2749. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45103-X_50.
   https://doi.org/10.1007/3-540-45103-X_50
10. Bay, H., Ess, A., Tuytelaars, T., & Van Gool, L., 2008. “Speeded-up robust features (SURF)”. Computer vision and image understanding, 110(3), pp. 346-359, https://doi.org/10.1016/j.cviu.2007.09.014.
11. Hartley, R., Zisserman, A., 2004. Multiple View Geometry in Computer Vision. Cambridge University Press, 2 edn. https://doi.org/10.1017/CBO9780511811685
12. Lowe, D.G., 2004. “Distinctive image features from scale-invariant key points”. Int. J. Comput. Vision 60(2), pp. 91-110. https://doi.org/10.1023/B:VISI.0000029664.99615.94
13. Rublee, E., Rabaud, V., Konolige, K., Bradski, G., 2011. “ORB: An efficient alternative to SIFT or SURF”. In International conference on computer vision. pp. 2564-2571. IEEE Computer Society, http://dblp.uni-trier.de/db/conf/iccv/iccv2011. html# Ru¬bleeRKB11.
    https://doi.org/10.1109/ICCV.2011.6126544
14. Fischler, M.A., Bolles, R.C., 1981. “Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography”. Commun. ACM, 24(6), pp. 381-395.
    https://doi.org/10.1145/358669.358692
15. Van der Walt, S., Schönberger, J.L., Nunez-Iglesias, J., Boulogne, F., Warner, J.D., Yager, N. et. all, 2014. Scikit-image: image processing in Python. PeerJ, 2, e453. https://doi.org/10.7717/peerj.453.

Received  14.12.2021