Evaluation of optical character recognition accuracy of Claude Opus 4 in handwritten student compositions

Authors : Yusuf Emre Yeşilyurt

Pages : 142-156

Doi:10.30783/nevsosbilen.1770463

View : 89 | Download : 536

Publication Date : 2025-12-07

Article Type : Research Paper

Abstract :This study evaluates Claude Opus 4 for optical character recognition (OCR) on 30 handwritten English as a Foreign Language (EFL) essay by testing if its fidelity is adequate for AI-supported assessment. For each essay, a human baseline and Claude’s verbatim output were obtained via the web interface. Accuracy was analyzed at character, word, and sentence-coherence levels. The findings revealed high character accuracy (over 98%) and strong word accuracy (over 96%), though sentence coherence was more variable. While word accuracy showed a significant correlation with coherence, character accuracy did not. A 95% word-accuracy threshold was found effective in separating usable from unreliable transcripts. The most common errors were substitutions, punctuation/capitalization, and deletions, while rarer hallucinated insertions (text not present in the original) and semantic distortions disproportionately harmed meaning. The study recommends spot-checking outputs with ≥95% word accuracy and fully re-transcribing those below, suggesting that an “AI + Teacher” workflow remains essential for high-stakes use.
Keywords : El Yazısı Tanıma, Otomatik Yazma Değerlendirmesi, Çok Kipli Büyük Dil Modelleri, Öğretmen–YZ İş Birliği

ORIGINAL ARTICLE URL