• P-ISSN 0974-6846 E-ISSN 0974-5645

Indian Journal of Science and Technology


Indian Journal of Science and Technology

Year: 2022, Volume: 15, Issue: 9, Pages: 386-394

Original Article

A Novel Fusion Attention Algorithm for Sentimental Image Analysis

Received Date:20 November 2021, Accepted Date:04 February 2022, Published Date:05 March 2022


Objectives: To implement a novel and hybrid methodology for finding out the positive features when using convolutional neural networks (CNNs) for visual sentiment analysis. To achieve increased accuracy, precision and recall by using this proposed fusion attention methodology. Methods: This study proposes a modified methodology encompassing spatial attention, channel attention as well as squeeze excitation modules. An enhanced approach on the basis of convolutional neural networks was used here which utilizes convolution operators by combining both spatial and channel-based data. Moreover, we have incorporated three considerations like spatial, channel as well as squeeze and excitation at various levels for attaining optimal results. Findings: The accuracy of the existing approaches was 59.88%, 60.06%, 59.28% and 62.89%, but the proposed fusion attention method showed increased accuracy of 64.15%. Similarly, the F1 score of existing approaches are 0.464804, 0.250164, 0.474129 and 0.2574, but the proposed method revealed increased F1 score of 0.512933. Furthermore, the proposed algorithm showed precision and recall of 0.560896 and 0.472526 which were better when compared with the existing approaches like Res-Target, Resnet50, Alexnet and VGG16. Novelty: The novel feature of this proposed fusion attention algorithm was that it incorporates a hybrid approach in which the image together with convolution passes through channel attention, spatial attention as well as squeeze and excitation so as to attain increased accuracy, but most of the existing approaches have used only channel attention and spatial attention modules. In this proposed method, the algorithm performs convolution in 64-bit, 128-bit and 256-bit respectively together in which the three attentions were interchanged in each convolution, which were not prevalent in the existing approaches.

Keywords: Fusion attention algorithm; Sentimental image analysis; Convolutional neural networks; Convolution and pooling; Deep neural network


  1. Ou H, Qing C, Xu X, Jin J. Multi-Level Context Pyramid Network for Visual Sentiment Analysis. Sensors. 2021;21(6):2136. Available from: https://dx.doi.org/10.3390/s21062136
  2. Yan W, Zhou L, Qian Z, Xiao L, Zhu H. Sentiment Analysis of Student Texts Using the CNN-BiGRU-AT Model. Scientific Programming. 2021;2021:1–9. Available from: https://dx.doi.org/10.1155/2021/8405623
  3. Jamatia A, Swamy SD, Gambäck B, Das A, Debbarma S. Deep Learning Based Sentiment Analysis in a Code-Mixed English-Hindi and English-Bengali Social Media Corpus. International Journal on Artificial Intelligence Tools. 2020;29(05):2050014. Available from: https://dx.doi.org/10.1142/s0218213020500141
  4. Chen J, Mao Q, Xue L. Visual Sentiment Analysis With Active Learning. IEEE Access. 2020;8(1):185899–185908. Available from: https://dx.doi.org/10.1109/access.2020.3024948
  5. Ortis A, Farinella GM, Battiato S. A Survey on Visual Sentiment Analysis. . IET Image Processing. 2020. Available from: https://doi.org/10.1049/iet-ipr.2019.1270
  6. Balakrishnan V, Shi Z, Law CL, Lim R, Teh LL, Fan Y. A deep learning approach in predicting products’ sentiment ratings: a comparative analysis. The Journal of Supercomputing. 2021. Available from: https://dx.doi.org/10.1007/s11227-021-04169-6
  7. Li B, Ren H, Jiang X, Miao F, Feng F, Jin L. SCEP—A New Image Dimensional Emotion Recognition Model Based on Spatial and Channel-Wise Attention Mechanisms. IEEE Access. 2021;9:25278–25290. Available from: https://dx.doi.org/10.1109/access.2021.3057373
  8. Chellam GH, Roseline V. Analysis of sentimental images using deep learning approach. Journal of Mathematical and Computational Science. 2021;11(5):5474–5486. Available from: https://doi.org/10.28919/jmcs/5835
  9. Kabiito D, Nakatumba-Nabende J. Targeted Aspect-Based Sentiment Analysis for Ugandan Telecom Reviews from Twitter. Transactions on Computational Science and Computational Intelligence. 2021;p. 311–322. Available from: https://doi.org/10.1007/978-3-030-70296-0_24
  10. Jiang T, Hu Xj, Yao Xh, Tu Lp, Huang Jb, Ma Xx, et al. Tongue image quality assessment based on a deep convolutional neural network. BMC Medical Informatics and Decision Making. 2021;21(1):147. Available from: https://dx.doi.org/10.1186/s12911-021-01508-8
  11. Mhamed M, Sutcliffe R, Sun X, Feng J, Almekhlafi E, Retta EA. Improving Arabic Sentiment Analysis Using CNN-Based Architectures and Text Preprocessing. Computational Intelligence and Neuroscience. 2021;2021:1–12. Available from: https://dx.doi.org/10.1155/2021/5538791
  12. Wu BX, Yang CG, Zhong JP. Research on Transfer Learning of Vision-based Gesture Recognition. International Journal of Automation and Computing. 2021;18(3):422–431. Available from: https://dx.doi.org/10.1007/s11633-020-1273-9
  13. Abdelgwad M, Soliman A, Taloba A, Farghaly MF. Arabic aspect based sentiment analysis using bidirectional GRU based models. Journal of King Saud University - Computer and Information Sciences. 2021. Available from: https://dx.doi.org/10.1016/j.jksuci.2021.08.030
  14. Tuan TQ, Hady WL, Martin A, Naoko N. Visual Sentiment Analysis for Review Images with Item-Oriented and User-Oriented CNN: Reproducibility Companion Paper. Proceedings of the 28th ACM International Conference on Multimedia. 2020;p. 4444–4447. Available from: https://ink.library.smu.edu.sg/sis_research/5955
  15. Xia K, Zhou Q, Jiang Y, Chen B, Gu X. Deep residual neural network based image enhancement algorithm for low dose CT images. Multimedia Tools and Applications. 2021. Available from: https://dx.doi.org/10.1007/s11042-021-11024-6
  16. Nguyen G, Dlugolinsky S, Bobák M, Tran V, García ÁL, Heredia I, et al. Machine Learning and Deep Learning frameworks and libraries for large-scale data mining: a survey. Artificial Intelligence Review. 2019;52(1):77–124. Available from: https://dx.doi.org/10.1007/s10462-018-09679-z
  17. Scheidegger F, Istrate R, Mariani G, Benini L, Bekas C, Malossi C. Efficient image dataset classification difficulty estimation for predicting deep-learning accuracy. The Visual Computer. 2021;37(6):1593–1610. Available from: https://dx.doi.org/10.1007/s00371-020-01922-5
  18. Yu J, Jiang J, Xia R. Entity-Sensitive Attention and Fusion Network for Entity-Level Multimodal Sentiment Classification. IEEE/ACM Transactions on Audio, Speech, and Language Processing. 2020;28:429–439. Available from: https://dx.doi.org/10.1109/taslp.2019.2957872
  19. Zhao P, Zhang J, Fang W, Deng S. SCAU-Net: Spatial-Channel Attention U-Net for Gland Segmentation. Frontiers in Bioengineering and Biotechnology. 2020;8. Available from: https://dx.doi.org/10.3389/fbioe.2020.00670
  20. Alzubaidi L, Zhang J, Humaidi AJ, Al-Dujaili A, Duan Y, Al-Shamma O, et al. Review of deep learning: concepts, CNN architectures, challenges, applications, future directions. Journal of Big Data. 2021;8(1). Available from: https://dx.doi.org/10.1186/s40537-021-00444-8


© 2022 Roseline & Chellam. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Published By Indian Society for Education and Environment (iSee)


Subscribe now for latest articles and news.