• P-ISSN 0974-6846 E-ISSN 0974-5645

Indian Journal of Science and Technology


Indian Journal of Science and Technology

Year: 2023, Volume: 16, Issue: 41, Pages: 3691-3703

Original Article

Performance Evaluation of Feature Fusion Approaches for Indian Sign Language Recognition System

Received Date:14 July 2023, Accepted Date:12 September 2023, Published Date:12 November 2023


Objectives: To recognize and analyze the Indian sign language (ISL) gestures in simple background using various features and Machine learning classifiers. Methods: a) Data pre-processing: Contour matching approach for hand region segmentation b) Feature extraction: Local features like gradient & key point descriptors were extracted using HOG, SIFT, SURF, LBP, FAST, feature fusion is done by concatenating features of HOG with LBP, SIFT with FAST, BOVW model with SURF. c) Model development: SVM, Random Forest, Logistic regression, Naïve Bayes were trained on large dataset and was experimented with hyper parameter tuning. The experiment was performed on 2 standard datasets which consist of alpha numerals (A-Z & 1-9) in simple black background. Findings: Our research demonstrates that our model consistently achieved a remarkable 100% accuracy rate when utilizing feature fusion techniques on both of the datasets employed in this study. The research findings underscore the need to consider a more comprehensive approach for gesture recognition. Relying solely on distinct features extracted from a single algorithm is shown to be insufficient in addressing the challenges posed by the diverse nature of sign shapes, varying illumination conditions, and different orientations. This emphasizes the importance of exploring hybrid or multi-algorithmic strategies to enhance the accuracy and robustness of gesture recognition systems. Novelty: This research introduces a novel perspective on ISL gesture recognition by emphasizing gestures against a simple black background.  The innovative application of feature fusion, combining various elements like hand shape and orientation enhances accuracy. The inclusion of hand segmentation using contour matching algorithm and experimentation on benchmark data adds another layer of novelty, highlighting practical applicability. The experimental results show that SVM has given better results when used with different combinations of feature extractors.

Keywords: ISL, Feature Fusion, Keypoint Descriptors, HOG, SIFT, SURF, LBP, FAST


  1. Bhattacharya A, Zope V, Kumbhar K, Borwankar P, Mendes A. Classification of Sign Language Gestures using Machine Learning. International Journal of Advanced Research in Computer and Engineering. 2020;8(12):97–103. Available from: https://doi.org/10.17148/IJARCCE.2019.81219
  2. Kumbhar S, Landge A, Kulkarni A, Solanki D, Kurtadikar V, Karad V. Indian Sign Language Recognition System. International Journal of Innovative Science and Research Technology. 2021;6(6). Available from: https://ijisrt.com/assets/upload/files/IJISRT21JUN1179.pdf
  3. Mariappan HM, Gomathi V. Real-Time Recognition of Indian Sign Language. 2019 International Conference on Computational Intelligence in Data Science (ICCIDS). 2019. Available from: https://doi.org/10.1109/ICCIDS.2019.8862125
  4. Subramanian B, Olimov B, Naik SM, Kim S, Park KHH, Kim J. An integrated mediapipe-optimized GRU model for Indian sign language recognition. Scientific Reports. 2022;12(1):1–16. Available from: https://doi.org/10.1038/s41598-022-15998-7
  5. Pala G, Jethwani JB, Kumbhar SS, Patil SD. Machine Learning-based Hand Sign Recognition. 2021 International Conference on Artificial Intelligence and Smart Systems (ICAIS). 2021;p. 356–363. Available from: https://doi.org/10.1109/ICAIS50930.2021.9396030
  6. Sharma A, Mittal A, Singh S, Awatramani V. Hand Gesture Recognition using Image Processing and Feature Extraction Techniques. Procedia Computer Science. 2020;173:181–190. Available from: https://doi.org/10.1016/J.PROCS.2020.06.022
  7. Dhivyasri S, Hari KB, Akash M, Sona M, Divyapriya S, Krishnaveni V. An efficient approach for interpretation of Indian sign language using machine learning. 2021 3rd International Conference on Signal Processing and Communication (ICPSC). 2021;2021:130–133. Available from: https://doi.org/10.1109/ICSPC51351.2021.9451692
  8. Badhe PC, Kulkarni V. Artificial Neural Network based Indian Sign Language Recognition using hand crafted features. 2020 11th International Conference on Computing, Communication and Networking Technologies (ICCCNT). 2020;p. 1–6. Available from: https://doi.org/10.1109/ICCCNT49239.2020.9225294
  9. Amrutha K, Prabu P. ML Based Sign Language Recognition System. 2021 International Conference on Innovative Trends in Information Technology (ICITIIT). 2021. Available from: https://doi.org/10.1109/ICITIIT51526.2021.9399594
  10. Katoch S, Singh V, Tiwary US. Indian Sign Language recognition system using SURF with SVM and CNN. Array. 2022;14:100141. Available from: https://doi.org/10.1016/J.ARRAY.2022.100141
  11. Munnaluri V, Pandey V, Singh P. Machine Learning based Approach for Indian Sign Language Recognition. 2022 7th International Conference on Communication and Electronics Systems (ICCES). 2022;p. 1128–1132. Available from: https://doi.org/10.1109/ICCES54183.2022.9835908
  12. Sreemathy R, Turuk M, Kulkarni I, Khurana S. Sign language recognition using artificial intelligence. Education and Information Technologies. 2023;28(5):5259–5278. Available from: https://doi.org/10.1007/s10639-022-11391-z
  13. Nguyen HBD, Do HN. Deep Learning for American Sign Language Fingerspelling Recognition System. 2019 26th International Conference on Telecommunications (ICT). 2019;p. 314–318. Available from: https://doi.org/10.1109/ICT.2019.8798856
  14. Arulkumar V, Prakash SJ, Subramanian EK, Thangadurai N. An Intelligent Face Detection by Corner Detection using Special Morphological Masking System and Fast Algorithm. 2021 2nd International Conference on Smart Electronics and Communication (ICOSEC). 2021;p. 1556–1561. Available from: https://doi.org/10.1109/ICT.2019.8798856
  15. Dhivyasri S, Hari KB, Akash M, Sona M, Divyapriya S, Krishnaveni V. An efficient approach for interpretation of Indian sign language using machine learning. 2021 3rd International Conference on Signal Processing and Communication (ICPSC). 2021;2021:130–133. Available from: https://doi.org/10.1109/ICSPC51351.2021.9451692
  16. Shravani K, Lakshmi A, Geethika S, Sapna B. Indian Sign Language Character Recognition. IOSR Journal of Computer Engineering. 2020;22(3):14–19. Available from: https://www.iosrjournals.org/iosr-jce/papers/Vol22-issue3/Series-1/B2203011419.pdf
  17. Manikandan J, Krishna BV, Narayan SS, Surendar K. Sign Language Recognition using Machine Learning. 2022 International Conference on Innovative Computing, Intelligent Communication and Smart Electrical Systems (ICSES). 2022;p. 3235–3240. Available from: https://doi.org/10.1109/ICSES55317.2022.9914155
  18. Sahoo AK. Indian Sign Language Recognition Using Machine Learning Techniques. Macromolecular Symposia. 2021;397(1). Available from: https://doi.org/10.1002/masy.202000241
  19. Joshi G, Singh S, Vig R. Taguchi-TOPSIS based HOG parameter selection for complex background sign language recognition. Journal of Visual Communication and Image Representation. 2020;71:102834. Available from: https://doi.org/10.1016/j.jvcir.2020.102834


© 2023 Poornima et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Published By Indian Society for Education and Environment (iSee)


Subscribe now for latest articles and news.