Frequency and Spatial Domain-Based Approaches for Recognition of Indian Sign Language Gestures

B V Poornima; S Srinath

doi:10.17485/IJST/v17i7.2836

Article

Frequency and Spatial Domain-Based Approaches for Recognition of Indian Sign Language Gestures

VIEWS 143
PDF 47

Indian Journal of Science and Technology

DOI: 10.17485/IJST/v17i7.2836

Year: 2024, Volume: 17, Issue: 7, Pages: 660-669

Original Article

Frequency and Spatial Domain-Based Approaches for Recognition of Indian Sign Language Gestures

B V Poornima^1*, S Srinath¹

¹Department of CSE, SJCE, JSS Science and Technology University, Mysuru, Karnataka, India

*Corresponding Author
Email: [email protected]

Received Date:09 November 2023, Accepted Date:23 December 2023, Published Date:14 February 2024

This work is licensed under a Creative Commons Attribution 4.0 International License.

Abstract

Objectives: The objective of this paper is to introduce and demonstrate an innovative approach for the recognition of Indian sign language gestures, with a focus on bridging communication gap between the deaf and hearing communities. The goal is to contribute to the development of effective tools and technologies that facilitate seamless communication between individuals using sign language and the people with no knowledge about sign language. Methods: The methodology consists of three key steps. First, data pre-processing involves resizing and contours extraction. Next, feature extraction employs Fourier descriptors for frequency domain analysis and gray-level-co-occurrence matrix for spatial domain analysis. Finally, various machine learning models including SVM, Random Forest, Logistic Regression, K-Nearest Neighbor and Naive Bayes are trained on a standard dataset. Findings: In our controlled experimental setup, we applied a diverse set of machine learning classifiers to evaluate the proposed approach for gesture recognition. Among the classifiers tested, K-Nearest Neighbors demonstrated the highest accuracy, achieving 99.82%. To validate the robustness of our approach, we employed k-fold cross-validation with 5 folds. Novelty: This study presents an innovative method for sign language recognition by employing a dual-domain fusion strategy that prominently emphasizes the frequency domain. Through the integration of Fourier descriptors, the research conducts a detailed frequency domain analysis to characterize the contour shapes of sign language gestures. The synergy with gray-level co-occurrence matrix texture features in the spatial domain analysis, contributes to the creation of a comprehensive feature vector. The proposed approach ensures a thorough exploration of gesture features, there by advancing the precision and efficacy of sign language recognition.

Keywords: Indian Sign Language (ISL), Sign Language Recognition (SLR), Frequency domain, Spatial domain, Fourier descriptors, Gray level cooccurrence matrix (GLCM), K Fold

References

Vazquez-Enriquez M, Alba-Castro JL, Docio-Fernandez L, Rodriguez-Banga E. Isolated Sign Language Recognition with Multi-Scale Spatial-Temporal Graph Convolutional Networks. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). (pp. 3457-3466) IEEE. 2021.
Indra D, Purnawansyah, Madenda SP, Wibowo EP. Indonesian Sign Language Recognition Based on Shape of Hand Gesture. Procedia Computer Science. 2019;161:74–81. Available from: https://doi.org/10.1016/j.procs.2019.11.101
Yang C, Yu Q. Multiscale Fourier descriptor based on triangular features for shape retrieval. Signal Processing: Image Communication. 2019;71:110–119. Available from: https://doi.org/10.1016/j.image.2018.11.004
Mali D, Limkar N, Mali S. Indian Sign Language Recognition using SVM Classifier. In: International Conference on Communication and Information Processing (ICCIP-2019). (pp. 1-9) 2019.
Tavari NV, Deorankar AV. Indian Sign Language Recognition based on Histograms of Oriented Gradient. International Journal of Computer Science and Information Technologies. 2014;5(3):3657–3660. Available from: https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=b188e93f8aa17a6c415bb2e558ee973538510f25#:~:text=For%20the%20recognition%20of%20ISL,the%20purpose%20of%20object%20detection.
Tharwat G, Ahmed AM, Bouallegue B. Arabic Sign Language Recognition System for Alphabets Using Machine Learning Techniques. Journal of Electrical and Computer Engineering. 2021;2021:1–17. Available from: https://doi.org/10.1155/2021/2995851
Intwala N, Banerjee A, Meenakshi, Gala N. Indian Sign Language converter using Convolutional Neural Networks. In: 2019 IEEE 5th International Conference for Convergence in Technology (I2CT). (pp. 1-5) IEEE. 2020.
Wang ZZ. Gesture recognition by model matching of slope difference distribution features. Measurement. 2021;181:109590. Available from: https://doi.org/10.1016/j.measurement.2021.109590
Miao Y, Li J, Liu J, Chen J, Sun S. Hand Gesture Recognition Based on Joint Rotation Feature and Fingertip Distance Feature. Chinese Journal of Computers. 2020;1:78–92. Available from: https://caod.oriprobe.com/articles/58986786/Hand_Gesture_Recognition_Based_on_Joint_Rotation_F.htm
Dhiman C, Vishwakarma DK. A Robust Framework for Abnormal Human Action Recognition Using R -Transform and Zernike Moments in Depth Videos. IEEE Sensors Journal. 2019;19(13):5195–5203. Available from: https://doi.org/10.1109/JSEN.2019.2903645
Karthikeyu, Shravanikatte. Indian-sign-language-recognition. Available from: https://github.com/Karthikeyu/Indian-sign-language-recognition
Subudhi P, Mukhopadhyay S. A statistical active contour model for interactive clutter image segmentation using graph cut optimization. Signal Processing. 2021;184:108056. Available from: https://doi.org/10.1016/j.sigpro.2021.108056
Liu J, Yuan J, Cui J, Liu Y, Liu X. Contour Resampling-Based Garlic Clove Bud Orientation Recognition for High-Speed Precision Seeding. Agriculture. 2022;12(9):1–26. Available from: https://doi.org/10.3390/agriculture12091334
Khan US, Zhang X, Su Y. Active Contour Model Using Fast Fourier Transformation for Salient Object Detection. Electronics. 2021;10(2):1–20. Available from: https://doi.org/10.3390/electronics10020192
Kim T, Jung YH. Correlation of Roundness and Sphericity with Complex Fourier Descriptors of 2-Dimensional Projected Images of Soil Particles. KSCE Journal of Civil Engineering. 2022;26(3):1089–1094. Available from: https://doi.org/10.1007/s12205-021-5922-5
Singh J, Singh H, Goyal V. A Comprehensive Study on Feature Extraction and Classification Techniques for Sign Language Recognition. In: 2021 5th International Conference on Electronics, Communication and Aerospace Technology (ICECA). (pp. 959-964) IEEE. 2022.
Erwin, Karo NNB, Sari AY, Aziza N, Putra HK. The Enhancement of Fingerprint Images using Gabor Filter. In: International Conference on Information System, Computer Science and Engineering, Journal of Physics: Conference Series. (Vol. 1196, pp. 012045) IOP Publishing. 2019.
Indian Sign Language Recognition Using Canny Edge Detection. International Journal of Advanced Trends in Computer Science and Engineering. 2021;10(3):1576–1583. Available from: https://doi.org/10.30534/ijatcse/2021/131032021
Shravani K, Lakshmi AS, Geethika MS, Kulkarni SB. Indian Sign Language Character Recognition. IOSR Journal of Computer Engineering (IOSR-JCE). 2020;22(3):14–19. Available from: https://www.iosrjournals.org/iosr-jce/papers/Vol22-issue3/Series-1/B2203011419.pdf
Munnaluri V, Pandey V, Singh P. Machine Learning based Approach for Indian Sign Language Recognition. In: 2022 7th International Conference on Communication and Electronics Systems (ICCES). (pp. 1128-1132) IEEE. 2022.
Savant R, Nasriwala J, Bhatt P. Static Gesture Recognition for Indian Sign Language Alphabets and Numbers using SVM with ORB Keypoints and Image Pixel As Feature. International Journal of Engineering Research & Technology (IJERT). 2022;11(1):214–219. Available from: https://www.ijert.org/research/static-gesture-recognition-for-indian-sign-language-alphabets-and-numbers-using-svm-with-orb-keypoints-and-image-pixel-as-feature-IJERTV11IS010090.pdf

Copyright

© 2024 Poornima & Srinath. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Published By Indian Society for Education and Environment (iSee)