Comparison of Methods Used in Detection of DIF in Cognitive Diagnostic Models with Traditional Methods: Applications in TIMSS 2011

Authors : Büşra EREN, Tuba GÜNDÜZ, Şeref TAN

Pages : 76-94

Doi:10.21031/epod.1218144

View : 85 | Download : 19

Publication Date : 2023-03-25

Article Type : Research Paper

Abstract :This study aims to compare the Wald test and likelihood ratio test insert ignore into journalissuearticles values(LRT); approaches with Classical Test Theory insert ignore into journalissuearticles values(CTT); and Item Response Theory insert ignore into journalissuearticles values(IRT); based differential item functioning insert ignore into journalissuearticles values(DIF); detection methods in the context of cognitive diagnostic models insert ignore into journalissuearticles values(CDMs);, using the TIMSS 2011 dataset as a retrofitting study. CDMs, which have a significant potential when determining the DIF and their contribution to validity, can give confidence under the strong methodological background condition is met. Therefore, it is hoped that this study will contribute to the literature to ensure the correct usage of CDMs and evaluate the compatibility of these new approaches with traditional methods. According to the analysis results, thirty-one items showed differences between the cognitive diagnosis assessments and the traditional methods. The item with the largest DIF was found in the Raju Unsigned Area Measures technique in IRT, whereas the item with the lowest DIF was found in the Wald test technique developed for CDMs. In general, the analyses show that methods not based on CDMs detect more items with DIF, but the Wald test and LRT methods based on CDMs detect fewer items with DIF. This study conducted DIF analyses to determine the test\`s psychometric properties within the framework of CDMs rather than the source of the bias. Researchers can take the study one step further and make more specific assessments about the items\` bias regarding the test structure, test scope, and subgroups. In addition, DIF analyses in this study were carried out using only the gender variable, and researchers can use different variables to conduct studies specific to their purpose.
Keywords : Cognitive diagnosis models, conventional methods, differential item functioning

ORIGINAL ARTICLE URL