AI-Generated Content Detectors: Boon or Bane for Scientific Writing

Namrata Ladha; Kaushal Yadav; Priya Rathore

doi:10.17485/IJST/v16i39.1632

Article

AI-Generated Content Detectors: Boon or Bane for Scientific Writing

VIEWS 722
PDF 113

Indian Journal of Science and Technology

DOI: 10.17485/IJST/v16i39.1632

Year: 2023, Volume: 16, Issue: 39, Pages: 3435-3439

Original Article

AI-Generated Content Detectors: Boon or Bane for Scientific Writing

Namrata Ladha^1,2*, Kaushal Yadav¹, Priya Rathore³

¹Senior Research Fellow, Devi Ahilya Vishawavidyalaya, Devi Ahilya Vishawavidyalaya, Indore, M.P., India
²Executive Programme Student, Indian Institute of Management, Kozhikode, Kerala, India
³Junior Research Fellow, Devi Ahilya Vishawavidyalaya, Indore, M.P., India

*Corresponding Author
Email: [email protected]

Received Date:02 July 2023, Accepted Date:12 September 2023, Published Date:25 October 2023

This work is licensed under a Creative Commons Attribution 4.0 International License.

Abstract

Objectives: The purpose of the study is to help publishers identify AI-generated text in scientific research, academic works, and assignments as a critical step toward the regulation and promotion of the ethical usage of AI in academia. Method: Recently developed literature on Generative AI suggests that human reviewers may fail to distinguish between human and AI-generated articles. Therefore, the present study evaluates AI-powered software as a potential solution for AI-generated content detection. We performed an experiment to see whether AI detector tools are capable of identifying and distinguishing between human- and AI-generated texts. To determine the accuracy of the AI-detector, we created and submitted four research articles to AI-detector tools for a pre- and post- manipulation test. Findings: The study shows that it is quite impossible for any AI detector to identify all AI-generated content accurately, thus, human-AI collaboration strategies can be employed to achieve the maximum accuracy. This paper demonstrates in a novel manner how AI detector tools can be manipulated to provide false results. Novelty: To the best of the authors’ knowledge, this paper is the first to acknowledge growing AI-literacy among students and scholars. This study edifies academia by providing scientifically verified human-AI collaboration strategies to capitalize on these tools and thwart academic misconduct.

Keywords: Artificial Intelligence, Chatbot, ChatGPT, Generative AI, Scientific Research

References

Buriak JM, Akinwande D, Artzi N, Brinker CJ, Burrows C, Chan WC, et al. Best Practices for Using AI When Writing Scientific Manuscripts: Caution, Care, and Consideration: Creative Science Depends on It. ACS nano. 2023;17(5):4091–4093. Available from: https://doi.org/10.1021/acsnano.3c01544
Elali FR, Rachid LN. AI-generated research paper fabrication and plagiarism in the scientific community. Patterns. 2023;4(3):1–4. Available from: https://doi.org/10.1016/j.patter.2023.100706
Cao Y, Li S, Liu Y, Yan Z, Dai Y, Yu PS, et al. A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT. 2023. Available from: https://doi.org/10.48550/arXiv.2303.04226
Saxena A, Chang AYY, Kulathuramaiyer N, Bhatt A. The Conceptual Framework of Articulated Human Intelligence in Perspective of Industry 4.0. Indian Journal of Science and Technology. 2019;12(6):1–5. Available from: https://doi.org/10.17485/ijst/2019/v12i6/141147
Dwivedi YK, Kshetri N, Hughes L, Slade EL, Jeyaraj A, Kar AK, et al. Opinion Paper: “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management. 2023;71:1–63. Available from: https://doi.org/10.1016/j.ijinfomgt.2023.102642
Zheng H, Zhan H. ChatGPT in Scientific Writing: A Cautionary Tale. The American Journal of Medicine. 2023;136(8):725–726.E6. Available from: https://doi.org/10.1016/j.amjmed.2023.02.011
Eke DO. ChatGPT and the rise of generative AI: Threat to academic integrity? Journal of Responsible Technology. 2023;13:1–4. Available from: https://doi.org/10.1016/j.jrt.2023.100060
Indian Society for Education and Environment. Publication Policies. 2023. Available from: https://sciresol.s3.us-east-2.amazonaws.com/IJST/Policy/policy-and-conditions.pdf
ChatGPT and Artificial Intelligence in higher education. UNESCO. Available from: https://www.iesalc.unesco.org/wp-content/uploads/2023/04/ChatGPT-and-Artificial-Intelligence-in-higher-education-Quick-Start-guide_EN_FINAL.pdf
English Google SEO office-hours from April 1, 2022. Google search central. Available from: https://youtu.be/Yx997SsiYlw.Accessed (accessed 2023-May-10)
Anderson N, Belavy DL, Perle SM, Hendricks S, Hespanhol L, Verhagen E, et al. AI did not write this manuscript, or did it? Can we trick the AI text detector into generated texts? The potential future of ChatGPT and AI in Sports & Exercise Medicine manuscript generation. BMJ Open Sport & Exercise Medicine. 2023;9(1):1–4. Available from: http://dx.doi.org/10.1136/bmjsem-2023-001568
Cotton DRE, Cotton PA, Shipway JR. Chatting and cheating: Ensuring academic integrity in the era of ChatGPT. Innovations in Education and Teaching International . 2023;p. 1–12. Available from: https://doi.org/10.1080/14703297.2023.2190148
Susnjak T. ChatGPT: The End of Online Exam Integrity? 2022. Available from: https://doi.org/10.48550/arXiv.2212.09292
Gao CA, Howard FM, Markov NS, Dyer EC, Ramesh S, Luo YS, et al. Comparing scientific abstracts generated by ChatGPT to real abstracts with detectors and blinded human reviewers. npj Digital Medicine. 2023;6(1):1–5. Available from: https://doi.org/10.1038/s41746-023-00819-6
Cingillioglu I. Detecting AI-generated essays: the ChatGPT challenge. International Journal of Information and Learning Technology. 2023;40(3):259–268. Available from: https://doi.org/10.1108/IJILT-03-2023-0043
Weber-Wulff D, Anohina-Naumeca A, Bjelobaba S, Foltýnek T, Guerrero-Dib J, Popoola O, et al. Testing of Detection Tools for AI-Generated Text. 2023;p. 1–46. Available from: https://arxiv.org/abs/2306.15666
Walters WH. The Effectiveness of Software Designed to Detect AI-Generated Writing: A Comparison of 16 AI Text Detectors. Open Information Science. 2023;p. 1–27. Available from: https://www.researchgate.net/publication/373948577_The_effectiveness_of_software_designed_to_detect_AI-generated_writing_A_comparison_of_16_AI_text_detectors
8 Most Accurate and Reliable AI Content Detectors in 2023. 10web Blog. Available from: https://10web.io/blog/ai-detectors/Accessed
Copyleaks. Trusted by Leading Organizations. Available from: https://copyleaks.com/ (accessed 2023-May-14)
The AI writing platform of top teams. Writer. Available from: https://writer.com/customers/ (accessed 2023-May-10)
About us. Content at scale. Available from: https://contentatscale.ai/about-us/ (accessed 2023-May-14)
Professors are using ChatGPT detector tools to accuse students of cheating. But what if the software is wrong? USA Today. Available from: https://www.usatoday.com/story/news/education/2023/04/12/how-ai-detection-tool-spawned-false-cheating-case-uc-davis/11600777002/.AccessedMay10
Misinformation machines? AI chatbots can spew falsehoods, even accuse people of crimes they never committed. Fox News. Available from: https://www.foxnews.com/media/misinformation-machines-ai-chatbots-spew-falsehoods-even-accuse-people-crimes-never-committed.Accessed
Ai. New AI classifier for indicating AI-written text. Open AI. Available from: https://openai.com/blog/new-ai-classifier-for-indicating-ai-written-text

Copyright

© 2023 Ladha et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Published By Indian Society for Education and Environment (iSee)