• P-ISSN 0974-6846 E-ISSN 0974-5645

Indian Journal of Science and Technology


Indian Journal of Science and Technology

Year: 2024, Volume: 17, Issue: 15, Pages: 1586-1595

Original Article

A Framework for Video Summarization using Visual Attention Technique

Received Date:17 February 2024, Accepted Date:26 March 2024, Published Date:12 April 2024


Objectives: To develop an efficient Video Summarization technique that aims to utilize the saliency map for mimicking the human way of selecting the important events in the given video. Methods: This paper proposes Histogram based Weighted Fusion (HWF) algorithm that uses spatial and temporal saliency maps to act as guidance in creating the summary of the video. The spatial saliency score and temporal saliency score obtained from the corresponding saliency maps are fused using the proposed HWF algorithm to obtain the frame level importance score. It tries to depict the visual attention of the human brain when watching a particular video. Findings: The experimental results show that the proposed HWF algorithm performs better than the state-of-the-art methods. Novelty: The use of Histogram intersection and the incorporation of the exponential function as the weight for the combined feature enhance the summarization ability of the proposed model.

Keywords: Video Summarization, Saliency Map, Histogram intersection, Contrast sensitivity function, Attention curves


  1. Rani S, Kumar M. Social media video summarization using multi-Visual features and Kohnen's Self Organizing Map. Information Processing & Management. 2020;57(3):102190. Available from: https://dx.doi.org/10.1016/j.ipm.2019.102190
  2. Sasithradevi A, Roomi SMM. A new pyramidal opponent color-shape model based video shot boundary detection. Journal of Visual Communication and Image Representation. 2020;67:102754. Available from: https://dx.doi.org/10.1016/j.jvcir.2020.102754
  3. Li Wl, Zhang T, Liu X. A static video summarization approach via block-based self-motivated visual attention scoring mechanism. International Journal of Machine Learning and Cybernetics. 2023;14(9):2991–3002. Available from: https://dx.doi.org/10.1007/s13042-023-01814-9
  4. Gupta D, Sharma A. A comprehensive study of automatic video summarization techniques. Artificial Intelligence Review. 2023;56(10):11473–11633. Available from: https://dx.doi.org/10.1007/s10462-023-10429-z
  5. Zhong SH, Lin J, Lu J, Fares A, Ren T. Deep Semantic and Attentive Network for Unsupervised Video Summarization. ACM Transactions on Multimedia Computing, Communications, and Applications. 2022;18(2):1–21. Available from: https://dx.doi.org/10.1145/3477538
  6. Mounika BR, Prakash O, Khare A. Fusion of Zero-Normalized Pixel Correlation Coefficient and Higher-Order Color Moments for Keyframe Extraction. In: Recent Trends in Communication, Computing, and Electronics, Lecture Notes in Electrical Engineering. (Vol. 524, pp. 357-364) Singapore. Springer . 2018.
  7. Saini P, Kumar K, Kashid S, Saini A, Negi A. Video summarization using deep learning techniques: a detailed analysis and investigation. Artificial Intelligence Review. 2023;56(11):12347–12385. Available from: https://dx.doi.org/10.1007/s10462-023-10444-0
  8. Haq HBU, Asif M, Ahmad MB, Ashraf R, Mahmood T. An Effective Video Summarization Framework Based on the Object of Interest Using Deep Learning. Mathematical Problems in Engineering. 2022;2022:1–25. Available from: https://doi.org/10.1155/2022/7453744
  9. Muhammad W, Ahmed I, Ahmad J, Nawaz M, Alabdulkreem E, Ghadi Y. A video summarization framework based on activity attention modeling using deep features for smart campus surveillance system. PeerJ Computer Science. 2022;8:1–21. Available from: https://doi.org/10.7717/peerj-cs.911
  10. Cagliero L, Canale L, Farinetti L. Data-Driven Analysis of Student Engagement in Time-Limited Computer Laboratories. Algorithms. 2023;16(10):1–26. Available from: https://dx.doi.org/10.3390/a16100464
  11. Pramanik A, Pal SK, Maiti J, Mitra P. Traffic Anomaly Detection and Video Summarization Using Spatio-Temporal Rough Fuzzy Granulation With Z-Numbers. IEEE Transactions on Intelligent Transportation Systems. 2022;23(12):24116–24125. Available from: https://dx.doi.org/10.1109/tits.2022.3198595
  12. Hu W, Zhang Y, Li Y, Zhao J, Hu X, Cui Y, et al. Query-based video summarization with multi-label classification network. Multimedia Tools and Applications. 2023;82(24):37529–37549. Available from: https://dx.doi.org/10.1007/s11042-023-15126-1
  13. Puthige I, Hussain T, Gupta S, Agarwal M. Attention Over Attention: An Enhanced Supervised Video Summarization Approach. Procedia Computer Science. 2023;218:2359–2368. Available from: https://dx.doi.org/10.1016/j.procs.2023.01.211
  14. Chang X, Ren P, Xu P, Li Z, Chen X, Hauptmann A. A Comprehensive Survey of Scene Graphs: Generation and Application. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2023;45(1):1–26. Available from: https://dx.doi.org/10.1109/tpami.2021.3137605
  15. Li M, Huang PY, Chang X, Hu J, Yang Y, Hauptmann A. Video Pivoting Unsupervised Multi-Modal Machine Translation. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2022;45(3):3918 –3932. Available from: https://dx.doi.org/10.1109/tpami.2022.3181116
  16. Zhang L, Chang X, Liu J, Luo M, Li Z, Yao L, et al. TN-ZSTAD: Transferable Network for Zero-Shot Temporal Activity Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2022;45(3):3848 –3861. Available from: https://dx.doi.org/10.1109/tpami.2022.3183586
  17. Sahu A, Chowdhury AS. First person video summarization using different graph representations. Pattern Recognition Letters. 2021;146:185–192. Available from: https://dx.doi.org/10.1016/j.patrec.2021.03.013
  18. Harakannanavar SS, Sameer SR, Kumar V, Behera SK, Amberkar AV, Puranikmath VI. Robust video summarization algorithm using supervised machine learning. Global Transitions Proceedings. 2022;3(1):131–135. Available from: https://dx.doi.org/10.1016/j.gltp.2022.04.009
  19. Alam I, Jalan D, Shaw P, Mohanta PP. Motion Based Video Skimming. In: 2020 IEEE Calcutta Conference (CALCON). IEEE. 2020.
  20. Ma M, Mei S, Wan S, Wang Z, Feng DD, Bennamoun M. Similarity Based Block Sparse Subset Selection for Video Summarization. IEEE Transactions on Circuits and Systems for Video Technology. 2021;31(10):3967–3980. Available from: https://dx.doi.org/10.1109/tcsvt.2020.3044600


© 2024 Dhanushree et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Published By Indian Society for Education and Environment (iSee)


Subscribe now for latest articles and news.