Identifying the Careless Responders: A Cross-Country Comparison on PISA 2018 Dataset

Authors : Başak Erdem Kara

Pages : 396-407

Doi:10.24106/kefdergi.1525320

View : 52 | Download : 62

Publication Date : 2024-07-31

Article Type : Research Paper

Abstract :Careless responding (CR) is one of the major concerns in self-report measures since it affects the quality of collected data. In this research, it was aimed to examine the dataset in terms of CR, to investigate the effects of CR on the psychometric properties of data, and to examine the effectiveness of CR detection methods in a cross-cultural context. Specifically; response time, outlier and long-string indices were used to flag CR and efficiency of using these were compared on PISA 2018 reading related attitude scales for Singapore and Turkey. As a result, it was indicated that the amount of CR was higher for Singapore than Turkey at long-string and response time methods. Furthermore, after removal of CR from dataset, enjoyment level of reading, perception of competence and perception of difficulty of PISA tests scores increased. Another result was that long-string and response time analyses provided significant improvement in model fit and long-string had the highest improvement level. Moreover, removing respondents according to outlier analyses didn’t provide any improvement on model-data fit. In general, results illustrated that careless responding behavior may have important effects on psychometric properties of self-report and screening the data for CR is strongly suggested in future studies.
Keywords : Careless Responding, Insufficient Effort Responding, Unmotivated Responding, PISA 2018, Self report Data