Evaluating the Accuracy of AI Content Detectors
Authors :Fatih Emre YILDIZ, Enis KARAARSLAN, Ömer AYDIN
Pages :143-155
Abstract :AI tools have become very popular for creating text, but they have also raised concerns about originality. To solve this, AI detectors are used to check if a text is written by a machine or a human. However, these tools are not always accurate. Literature was reviewed to understand how these AI content detectors work and the underlying methodologies. The current study will test five popular AI content detectors with different types of text: AI-generated text, famous scientific papers written and published before the LLMs, and other works written by humans. This will show us how these tools work well and where they fail. This article will also discuss examples of cases where AI detectors were used to incriminate people in academic or professional misconduct, to show what can happen if someone relies on these tools. From the results of this study, it was depicted that AI detectors often make misclassifications, confusing a human-written text with an AI-generated text. Results prove that the AI detectors used are not reliable 100%, and there needs to be some development. The findings further suggest that AI detection should not be used alone to make decisions, especially in high-stakes situations, as this can lead to unfair outcomes.Keywords :AI detection, Text originality, LLM, Academic integrity, Ai generator