A Framework for Video Summarization using Visual Attention Technique

M Dhanushree; R Priya; P Aruna; R Bhavani

doi:10.17485/IJST/v17i15.456

Article

A Framework for Video Summarization using Visual Attention Technique

VIEWS 76
PDF 41

Indian Journal of Science and Technology

DOI: 10.17485/IJST/v17i15.456

Year: 2024, Volume: 17, Issue: 15, Pages: 1586-1595

Original Article

A Framework for Video Summarization using Visual Attention Technique

M Dhanushree^1*, R Priya², P Aruna², R Bhavani³

¹Research scholar, Department of Computer Science and Engineering, Annamalai University, Chidambaram, Tamil Nadu, India
²Professor, Department of Computer Science and Engineering, Annamalai University, Chidambaram, Tamil Nadu, India
³Professor and Head, Department of Computer Science and Engineering, Annamalai University, Chidambaram, Tamil Nadu, India

*Corresponding Author
Email: [email protected]

Received Date:17 February 2024, Accepted Date:26 March 2024, Published Date:12 April 2024

This work is licensed under a Creative Commons Attribution 4.0 International License.

Abstract

Objectives: To develop an efficient Video Summarization technique that aims to utilize the saliency map for mimicking the human way of selecting the important events in the given video. Methods: This paper proposes Histogram based Weighted Fusion (HWF) algorithm that uses spatial and temporal saliency maps to act as guidance in creating the summary of the video. The spatial saliency score and temporal saliency score obtained from the corresponding saliency maps are fused using the proposed HWF algorithm to obtain the frame level importance score. It tries to depict the visual attention of the human brain when watching a particular video. Findings: The experimental results show that the proposed HWF algorithm performs better than the state-of-the-art methods. Novelty: The use of Histogram intersection and the incorporation of the exponential function as the weight for the combined feature enhance the summarization ability of the proposed model.

Keywords: Video Summarization, Saliency Map, Histogram intersection, Contrast sensitivity function, Attention curves

References

Rani S, Kumar M. Social media video summarization using multi-Visual features and Kohnen's Self Organizing Map. Information Processing & Management. 2020;57(3):102190. Available from: https://dx.doi.org/10.1016/j.ipm.2019.102190
Taylor W, Qureshi FZ. Real-time Video Summarization on Commodity Hardware. In: Proceedings of the 12th International Conference on Distributed Smart Cameras. (pp. 1-8) ACM. 2018.
Sasithradevi A, Roomi SMM. A new pyramidal opponent color-shape model based video shot boundary detection. Journal of Visual Communication and Image Representation. 2020;67:102754. Available from: https://dx.doi.org/10.1016/j.jvcir.2020.102754
Li Wl, Zhang T, Liu X. A static video summarization approach via block-based self-motivated visual attention scoring mechanism. International Journal of Machine Learning and Cybernetics. 2023;14(9):2991–3002. Available from: https://dx.doi.org/10.1007/s13042-023-01814-9
Gupta D, Sharma A. A comprehensive study of automatic video summarization techniques. Artificial Intelligence Review. 2023;56(10):11473–11633. Available from: https://dx.doi.org/10.1007/s10462-023-10429-z
Zhong SH, Lin J, Lu J, Fares A, Ren T. Deep Semantic and Attentive Network for Unsupervised Video Summarization. ACM Transactions on Multimedia Computing, Communications, and Applications. 2022;18(2):1–21. Available from: https://dx.doi.org/10.1145/3477538
Mounika BR, Prakash O, Khare A. Fusion of Zero-Normalized Pixel Correlation Coefficient and Higher-Order Color Moments for Keyframe Extraction. In: Recent Trends in Communication, Computing, and Electronics, Lecture Notes in Electrical Engineering. (Vol. 524, pp. 357-364) Singapore. Springer . 2018.
Saini P, Kumar K, Kashid S, Saini A, Negi A. Video summarization using deep learning techniques: a detailed analysis and investigation. Artificial Intelligence Review. 2023;56(11):12347–12385. Available from: https://dx.doi.org/10.1007/s10462-023-10444-0
Haq HBU, Asif M, Ahmad MB, Ashraf R, Mahmood T. An Effective Video Summarization Framework Based on the Object of Interest Using Deep Learning. Mathematical Problems in Engineering. 2022;2022:1–25. Available from: https://doi.org/10.1155/2022/7453744
Muhammad W, Ahmed I, Ahmad J, Nawaz M, Alabdulkreem E, Ghadi Y. A video summarization framework based on activity attention modeling using deep features for smart campus surveillance system. PeerJ Computer Science. 2022;8:1–21. Available from: https://doi.org/10.7717/peerj-cs.911
Cagliero L, Canale L, Farinetti L. Data-Driven Analysis of Student Engagement in Time-Limited Computer Laboratories. Algorithms. 2023;16(10):1–26. Available from: https://dx.doi.org/10.3390/a16100464
Pramanik A, Pal SK, Maiti J, Mitra P. Traffic Anomaly Detection and Video Summarization Using Spatio-Temporal Rough Fuzzy Granulation With Z-Numbers. IEEE Transactions on Intelligent Transportation Systems. 2022;23(12):24116–24125. Available from: https://dx.doi.org/10.1109/tits.2022.3198595
Shambharkar PG, Goel R. From video summarization to real time video summarization in smart cities and beyond: A survey. Frontiers in Big Data. 2022;5:1–14. Available from: https://doi.org/10.3389/fdata.2022.1106776
Hu W, Zhang Y, Li Y, Zhao J, Hu X, Cui Y, et al. Query-based video summarization with multi-label classification network. Multimedia Tools and Applications. 2023;82(24):37529–37549. Available from: https://dx.doi.org/10.1007/s11042-023-15126-1
Puthige I, Hussain T, Gupta S, Agarwal M. Attention Over Attention: An Enhanced Supervised Video Summarization Approach. Procedia Computer Science. 2023;218:2359–2368. Available from: https://dx.doi.org/10.1016/j.procs.2023.01.211
Chang X, Ren P, Xu P, Li Z, Chen X, Hauptmann A. A Comprehensive Survey of Scene Graphs: Generation and Application. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2023;45(1):1–26. Available from: https://dx.doi.org/10.1109/tpami.2021.3137605
Li M, Huang PY, Chang X, Hu J, Yang Y, Hauptmann A. Video Pivoting Unsupervised Multi-Modal Machine Translation. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2022;45(3):3918 –3932. Available from: https://dx.doi.org/10.1109/tpami.2022.3181116
Zhang L, Chang X, Liu J, Luo M, Li Z, Yao L, et al. TN-ZSTAD: Transferable Network for Zero-Shot Temporal Activity Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2022;45(3):3848 –3861. Available from: https://dx.doi.org/10.1109/tpami.2022.3183586
Sahu A, Chowdhury AS. First person video summarization using different graph representations. Pattern Recognition Letters. 2021;146:185–192. Available from: https://dx.doi.org/10.1016/j.patrec.2021.03.013
Harakannanavar SS, Sameer SR, Kumar V, Behera SK, Amberkar AV, Puranikmath VI. Robust video summarization algorithm using supervised machine learning. Global Transitions Proceedings. 2022;3(1):131–135. Available from: https://dx.doi.org/10.1016/j.gltp.2022.04.009
Mahasseni B, Lam M, Todorovic S. Unsupervised Video Summarization with Adversarial LSTM Networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). (pp. 202-211) IEEE. 2017.
Alam I, Jalan D, Shaw P, Mohanta PP. Motion Based Video Skimming. In: 2020 IEEE Calcutta Conference (CALCON). IEEE. 2020.
Ma M, Mei S, Wan S, Wang Z, Feng DD, Bennamoun M. Similarity Based Block Sparse Subset Selection for Video Summarization. IEEE Transactions on Circuits and Systems for Video Technology. 2021;31(10):3967–3980. Available from: https://dx.doi.org/10.1109/tcsvt.2020.3044600

Copyright

© 2024 Dhanushree et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Published By Indian Society for Education and Environment (iSee)