• P-ISSN 0974-6846 E-ISSN 0974-5645

Indian Journal of Science and Technology

Article

Delving into the Depths of Image Retrieval Systems in the Light of Deep Learning: A Review
  • VIEWS 283
  • PDF 82

Indian Journal of Science and Technology

DOI: 10.17485/IJST/v16i34.1341

Year: 2023, Volume: 16, Issue: 34, Pages: 2693-2702

Original Article

Delving into the Depths of Image Retrieval Systems in the Light of Deep Learning: A Review

Ovais Rashid Khan1*, Javaid Iqbal Bhat1

1Department of Computer Science, Islamic University of Science and Technology, Kashmir, India

*Corresponding Author
Email: [email protected]

Received Date:02 June 2023, Accepted Date:02 August 2023, Published Date:08 September 2023

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Abstract

Objective: The objective of this study is to conduct a comprehensive review of existing research and literature in the field of Content-Based Image Retrieval (CBIR). This review highlights the key challenges associated with the extraction and representation of visual semantics of images. This paper discusses the measure used computing similarity and ranking of retrieved images by CBIR system. The review discusses limitation of traditional approaches and also highlights the challenges with the current deep learning methods in semantic feature representation, defining the similarity metrics and indexing. This paper also highlights scalability and generalization challenges in implementing real environment. Methods: A thorough literature review was conducted on wellestablished databases, including Scopus, Web of Science, IEEE Xplore, ACM, and Science Direct, employing appropriate keywords. Mention the period of coverage. Pertinent search terms encompassed local feature representation, global feature representation, low-level features, high level features, semantic gap, image embeddings, handcrafted features, deep learning, image descriptors, similarity, and image indexing, with the aim of exploring content-based image retrieval systems. Comparative analysis was performed on the chosen articles, taking into account factors such as algorithms, methodologies, datasets, and evaluation metrics. The results discussed using comparative analysis, ensuring a comprehensive overview of recent literature on content-based image retrieval, offering valuable insights and highlighting emerging trends in the field. Findings : The research uncovers the novelty in the realm of contentbased image retrieval (CBIR) by highlighting the challenge of high-level visual semantics when comparing images, as perceived by humans. It emphasizes that feature extraction methods and choices significantly influence CBIR system performance, stressing the importance of selecting suitable features and similarity measures based on image dataset characteristics and application requirements. The study underscores the persistent obstacle of the semantic gap between low-level visual features and high-level semantic concepts, encouraging exploration of diverse approaches like deep learning, relevance feedback, and ontology-based methods to bridge this gap. Particularly, deep learn-ing techniques, notably Convolutional Neural Networks (CNNs), have shown promising results in CBIR by automatically learning hierarchical representations capturing high-level semantic information. However, the review also highlights the challenges of scaling deep learning methods and the limited accessibility of precisely labelled datasets, which can hinder performance and generalization across diverse image datasets and real-world scenarios. Deep learning models pose interpretability challenges due to their complex, opaque nature and hierarchical semantic representations.

Keywords: CBIR; ContentBased Image Retrieval; Deep Learning; Convolutional Neural Network; Local Features; Global Features; Similarity Metric; Semantic Gap

References

  1. Kumar RB, Marikkannu P. An Efficient Content Based Image Retrieval using an Optimized Neural Network for Medical Application. Multimedia Tools and Applications. 2020;79(31-32):22277–22292. Available from: https://doi.org/10.1007/s11042-020-08953-z
  2. Silva W, Gonçalves T, Härmä K, Schröder E, Obmann VC, Barroso MC, et al. Computer-aided diagnosis through medical image retrieval in radiology. Scientific Reports. 2022;12(1):20732. Available from: https://doi.org/10.1038/s41598-022-25027-2
  3. Babbar J, Rathee N. Satellite Image Analysis: A Review. 2019 IEEE International Conference on Electrical, Computer and Communication Technologies (ICECCT). 2019;p. 1–6. Available from: https://doi.org/10.1109/ICECCT.2019.8869481
  4. Wu Z, Jiang S, Zhou X, Wang Y, Zuo Y, Wu Z, et al. Application of image retrieval based on convolutional neural networks and Hu invariant moment algorithm in computer telecommunications. Computer Communications. 2020;150:729–738. Available from: https://doi.org/10.1016/j.comcom.2019.11.053
  5. Pondenkandath V, Alberti M, Eichenberger N, Ingold R, Liwicki M. Cross-Depicted Historical Motif Categorization and Retrieval with Deep Learning. Journal of Imaging. 2020;6(7):71. Available from: https://doi.org/10.3390/jimaging6070071
  8. Ibtihaal M, Hameed, Sadiq H, Abdulhussain, Basheera M, Mahmmod |DT, et al. Content-based image retrieval: A review of recent trends. Available from: https://doi.org/10.1080/23311916.2021.1927469
  9. Liu GH, Yang JY. Content-based image retrieval using color difference histogram. Pattern Recognition. 2013;46(1):188–198. Available from: https://doi.org/10.1016/j.patcog.2012.06.001
  10. Barbu T. Content-Based Image Retrieval Using Gabor Filtering. 2009 20th International Workshop on Database and Expert Systems Application. 2009;p. 236–240. Available from: https://doi.org/10.1109/DEXA.2009.61
  11. Chan YK, Tsai MH, Wang JS, Guo SW, Wu JL. Color-texture-based image retrieval system using Gaussian Markov random field model. Mathematical Problems in Engineering. 2009;p. 1–17. Available from: https://doi.org/10.1155/2009/410243
  12. Won CSW, Park DKP, Park SJPK. Efficient Use of MPEG-7 Edge Histogram Descriptor. ETRI Journal. 2002;24(1):23–30. Available from: https://doi.org/10.4218/etrij.02.0102.0103
  13. Zhang D, Lu G. Review of shape representation and description techniques. Pattern Recognition. 2004;37(1):1–19. Available from: https://doi.org/10.1016/j.patcog.2003.07.008
  14. Lowe DG. Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision. 2004;60(2):91–110. Available from: https://doi.org/10.1023/B:VISI.0000029664.99615.94
  15. Kalpana J, Krishnamoorthi R. Color image retrieval technique with local features based on orthogonal polynomials model and SIFT. Multimedia Tools and Applications. 2016;75(1):49–69. Available from: https://doi.org/10.1007/s11042-014-2262-1
  16. Alzu’bi A, Amira A, Ramzan N, Jaber T. Improving content-based image retrieval with compact global and local multi-features. International Journal of Multimedia Information Retrieval. 2016;5(4):237–253. Available from: https://doi.org/10.1007/s13735-016-0109-4
  17. Bay H, Ess A, Tuytelaars T, Gool LV. Speeded-Up Robust Features (SURF) Computer Vision and Image Understanding. 2008;110(3):346–359. Available from: https://doi.org/10.1016/j.cviu.2007.09.014
  19. Phadikar BS, Phadikar AS, Maity GK. Content-based image retrieval in DCT compressed domain with MPEG-7 edge descriptor and genetic algorithm. Pattern Analysis and Applications. 2018;21(2):469–489. Available from: https://doi.org/10.1007/s10044-016-0589-0
  20. Song W, Zhang Y, Liu F, Chai Z, Ding F, Qian X, et al. Taking advantage of multi-regions-based diagonal texture structure descriptor for image retrieval. Expert Systems with Applications. 2018;96:347–357. Available from: https://doi.org/10.1016/j.eswa.2017.12.006
  21. Raza A, Dawood H, Dawood H, Shabbir S, Mehboob R, Banjar A. Correlated Primary Visual Texton Histogram Features for Content Base Image Retrieval. IEEE Access. 2018;6:46595–46616. Available from: https://doi.org/10.1109/ACCESS.2018.2866091
  22. Jabeen S, Mehmood Z, Mahmood T, Saba T, Rehman A, Mahmood MT. An effective content-based image retrieval technique for image visuals representation based on the bag-of-visual-words model. PLOS ONE. 2018;13(4):e0194526. Available from: https://doi.org/10.1371/journal.pone.0194526
  23. Srivastava P, Khare A. Utilizing multiscale local binary pattern for content-based image retrieval. Multimedia Tools and Applications. 2018;77(10):12377–12403. Available from: https://doi.org/10.1007/s11042-017-4894-4
  24. Chen L, Li S, Bai Q, Yang J, Jiang S, Miao Y. Review of Image Classification Algorithms Based on Convolutional Neural Networks. Remote Sensing. 2021;13(22):4712. Available from: https://doi.org/10.3390/rs13224712
  25. Oerlemans A, Guo Y, Lew MS, Chua TS. Special issue on deep learning in image and video retrieval. International Journal of Multimedia Information Retrieval. 2020;9(2):61–62. Available from: https://doi.org/10.1007/s13735-020-00194-y
  26. Tao D, Tang X, Wu X. Asymmetric bagging and random subspace for support vector machines-based relevance feedback in image retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2006;28(7):1088–1099. Available from: https://doi.org/10.1109/TPAMI.2006.134
  27. Liu Y, Zhang D, Lu G. Region-based image retrieval with high-level semantics using decision tree learning. Pattern Recognition. 2008;41(8):2554–2570. Available from: https://doi.org/10.1016/j.patcog.2007.12.003
  28. Rajam IF, Valli S. Content-based image retrieval using a quick SVM-binary decision tree - QSVMBDT. Communications in Computer and Information Science. 2011;p. 11–22. Available from: https://doi.org/10.1007/978-3-642-24055-3_2
  29. Yousuf M, Mehmood Z, Habib HA, Mahmood T, Saba T, Rehman A, et al. A Novel Technique Based on Visual Words Fusion Analysis of Sparse Features for Effective Content-Based Image Retrieval. Mathematical Problems in Engineering. 2018;2018:1–13. Available from: https://doi.org/10.1155/2018/2134395
  30. Yuan X, Ren L, Lu J, Zhou J. Relaxation-Free Deep Hashing via Policy Gradient. Computer Vision – ECCV 2018. 2018;p. 141–157. Available from: https://doi.org/10.1007/978-3-030-01225-0_9
  31. Yang J, Zhang Y, Feng R, Zhang T, Fan W. Deep reinforcement hashing with redundancy elimination for effective image retrieval. Pattern Recognition. 2020;100:107116. Available from: https://doi.org/10.1016/j.patcog.2019.107116
  32. Yang W, Wang L, Cheng S. Deep parameter-free attention hashing for image retrieval. Scientific Reports. 2022;12(1):7082. Available from: https://doi.org/10.1038/s41598-022-11217-5
  33. Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Communications of the ACM. 2017;60(6):84–90. Available from: https://doi.org/10.1145/3065386
  35. Tzelepi M, Tefas A. Deep convolutional learning for Content Based Image Retrieval. Neurocomputing. 2018;275:2467–2478. Available from: https://doi.org/10.1016/j.neucom.2017.11.022
  37. Lu X, Zhang L, Niu L, Chen Q, Wang JQ. A Novel Adaptive Feature Fusion Strategy for Image Retrieval. Entropy. 2021;23(12):1670. Available from: https://doi.org/10.3390/e23121670
  38. Rashid Y, Bhat JI. Unlocking the Power of Social Networks with Community Detection Techniques for Isolated and Overlapped Communities: A Review. Indian Journal Of Science And Technology. 2023;16(25):1857–1871. Available from: https://doi.org/10.17485/IJST/v16i25.841

Copyright

© 2023 Khan & Bhat. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Published By Indian Society for Education and Environment (iSee)

  • 10 September 2023

Year: 2023, Volume: 16, Issue: 34

Article Metrics


Post Graduate Fellowship in Research Communication

More articles

DON'T MISS OUT!

Subscribe now for latest articles and news.