KHiTE: Multilingual Speech Acquisition to Monolingual Text Translation

Naveenkumar T Rudrappa; Mallamma V Reddy; M Hanumanthappa

doi:10.17485/IJST/v16i21.727

Article

KHiTE: Multilingual Speech Acquisition to Monolingual Text Translation

VIEWS 481
PDF 131

Indian Journal of Science and Technology

DOI: 10.17485/IJST/v16i21.727

Year: 2023, Volume: 16, Issue: 21, Pages: 1572-1579

Original Article

KHiTE: Multilingual Speech Acquisition to Monolingual Text Translation

Naveenkumar T Rudrappa^1*, Mallamma V Reddy², M Hanumanthappa³

¹Research Scholar, Department of Computer Science, Rani Channamma University, Belagavi, Karnataka, India
²Assistant Professor, Department of Computer Science, Rani Channamma University, Belagavi, Karnataka, India
³Professor of Computer Science and Applications, Jnanabharati Campus, Bangalore University, Bangalore, Karnataka, India

*Corresponding Author
Email: [email protected]

Received Date:28 March 2023, Accepted Date:06 May 2023, Published Date:31 May 2023

This work is licensed under a Creative Commons Attribution 4.0 International License.

Abstract

Objectives: To develop a system that accepts cross-lingual spoken reviews consisting of two to four languages, translate to target language text for Indic languages namely Kannada, Hindi, Telugu and/or English termed as cross lingual speech identification and text translation system. Methods: Hybridization of software engineering models are used in natural languages for pre-processing such as noise removal and speech splitting to obtain phonemes. Combinatorial models namely Hidden-Markov-Model, Artificial Neural Networks, Deep Neural Networks and Convulutional Neural Networks were deployed for direct and indirect speech mapping. Trained corpus consisting of thousand phonemes in the form of wave files for each language considered is named as KHiTEShabdanjali. The basic parameters cosidered for training dataset are pause, pitch, sampling frequency, threshold etc. Findings: The research has resulted in the development of mono-lingual and multi-lingual speech identification, tool for processing of cross-lingual speech and language identification, mono-lingual, bi-lingual, tri-lingual and quad-lingual speech to monolingual text translation for the four languages. It is a generic approach and can be used for other regional languages of India by training the corpus with the selected language. Novelty: Cross-lingual speech identification and text translation system helps users in e-shopping by reducing the time incurred in making a decision to purchase a product having enough features at an economical price, e-tutoring, e-farming activities, digitizing, defence etc.

Keywords: Artificial Neural Networks; Convulutional Neural Networks; Deep Neural Networks; HiddenMarkovModel; Speech Processing

References

Locke S, Bashall A, Al-Adely S, Moore J, Wilson A, Kitchen GB. Natural language processing in medicine: A review. Trends in Anaesthesia and Critical Care. 2021;38:4–9. Available from: https://doi.org/10.1016/j.tacc.2021.02.007
Boorugu R, Ramesh G. A Survey on NLP based Text Summarization for Summarizing Product Reviews. In: 2020 Second International Conference on Inventive Research in Computing Applications (ICIRCA). (pp. 352-356) IEEE. 2020.
Peguda J, Santosh VSS, Vijayalata Y, Mounish V. Speech to Sign Language Translation for Indian Languages. In: 2022 8th International Conference on Advanced Computing and Communication Systems (ICACCS). (pp. 1131-1135) IEEE. 2022.
Zue V, Glass J, Goodine D, Leung H, Phillips M, Polifroni J, et al. Integration of speech recognition and natural language processing in the MIT VOYAGER system. In: ICASSP 91: 1991 International Conference on Acoustics, Speech, and Signal Processing. (Vol. 91, pp. 713-716) IEEE. 1991.
Li S, You J, Zhang X. Overview and Analysis of Speech Recognition. In: 2022 IEEE International Conference on Advances in Electrical Engineering and Computer Applications (AEECA). (pp. 391-395) IEEE. 2022.
Zhu C, An K, Zheng H, Ou Z. Multilingual and Crosslingual Speech Recognition Using Phonological-Vector Based Phone Embeddings. In: 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). (pp. 1034-1041) IEEE. 2021.
Kumar PS, Jayanna HS. Performance Analysis of Hybrid Automatic Continuous Speech Recognition Framework for Kannada Dialect. In: 2019 10th International Conference on Computing, Communication and Networking Technologies (ICCCNT). (pp. 1-6) 2019.
Sharma RS, Paladugu SH, Priya KJ, Gupta D. Speech Recognition in Kannada using HTK and Julius: A Comparative Study. In: 2019 International Conference on Communication and Signal Processing (ICCSP). (p. 68) IEEE. 2019.
Dong L, Guo ZQ, Tan CH, Hu YJ, Jiang Y, Ling ZH. Neural Grapheme-To-Phoneme Conversion with Pre-Trained Grapheme Models. In: ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). (pp. 6202-6206) IEEE. 2022.
Khurana D, Koli A, Khatter K, Singh S. Natural language processing: state of the art, current trends and challenges. Multimedia Tools and Applications. 2023;82(3):3713–3744. Available from: https://doi.org/10.1007/s11042-022-13428-4
Zhang X, Peng Y, Xu X. An Overview of Speech Recognition Technology. In: 2019 4th International Conference on Control, Robotics and Cybernetics (CRC). (pp. 81-85) IEEE. 2019.
Brennan A, Mcdonagh T, Dempsey M, Mcavoy J. Cosmic Sounds: A Game to Support Phonological Awareness Skills for Children With Dyslexia. IEEE Transactions on Learning Technologies. 2022;15(3):301–310. Available from: https://doi.org/10.1109/TLT.2022.3170231
Ahmed T, Suffian M, Khan MY, Bogliolo A. Discovering Lexical Similarity Using Articulatory Feature-Based Phonetic Edit Distance. IEEE Access. 2022;10:1533–1544. Available from: https://doi.org/10.1109/ACCESS.2021.3137905
Chodroff E, Wilson C. Acoustic–phonetic and auditory mechanisms of adaptation in the perception of sibilant fricatives. Attention, Perception, & Psychophysics. 2020;82(4):2027–2048. Available from: https://doi.org/10.3758/s13414-019-01894-2
Romero D, D'haro LF, Estecha-Garitagoitia M, Salamea C. Phonotactic Language Recognition Using A Universal Phoneme Recognizer and A Transformer Architecture. In: ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). (pp. 6872-6876) IEEE. 2022.
N P, Kini S. Phoneme based Kannada Speech Corpus for Automatic Speech Recognition System. In: 2022 IEEE International Conference on Distributed Computing and Electrical Circuits and Electronics (ICDCECE). (pp. 1-5) 2022.
Yu F, Yao Z, Wang X, An K, Xie L, Ou Z, et al. The SLT 2021 Children Speech Recognition Challenge: Open Datasets, Rules and Baselines. In: 2021 IEEE Spoken Language Technology Workshop (SLT). (pp. 1117-1123) IEEE. 2021.
Xie S, Li L. Improvement and Application of Deep Belief Network Based on Sparrow Search Algorithm. In: 2021 IEEE International Conference on Advances in Electrical Engineering and Computer Applications (AEECA). (pp. 705-708) IEEE. 2021.
Nadimpalli VLV, Kesiraju S, Banka R, Kethireddy R, Gangashetty SV. Resources and Benchmarks for Keyword Search in Spoken Audio From Low-Resource Indian Languages. IEEE Access. 2022;10:34789–34799. Available from: https://doi.org/10.1109/ACCESS.2022.3162854
Black D, Rapos EJ, Stephan M. Voice-Driven Modeling: Software Modeling Using Automated Speech Recognition. In: 2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C). (pp. 252-258) IEEE. 2019.
Sulaimani SA, Starkey A. Short Text Classification Using Contextual Analysis. IEEE Access. 2021;9:149619–149629. Available from: https://doi.org/10.1109/ACCESS.2021.3125768
Rudrappa NT, Reddy MV, Hanumanthappa M. Standardized GUI Framework Using Python for Speech Processing: NLP. 2021 International Conference on Electrical, Computer and Energy Technologies (ICECET). 2021;p. 1–4. Available from: https://doi.org/10.1109/ICECET52533.2021.9698522
Rudrappa NT, Reddy MV, Hitek. HiTEK: Multilingual Speech Identification Using Combinatorial Model. In: International Conference On “Advances in Computer Vision and Artificial Intelligence Technologies ACVAIT. 2022.
Reddy MV, Rudrappa NT. Advanced Code Switching Application-COVID-19. SSRN Electronic Journal. 2020. Available from: http://dx.doi.org/10.2139/ssrn.3735927

Copyright

© 2023 Rudrappa et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Published By Indian Society for Education and Environment (iSee)