Comparison of Some Performance Metrics Used in Multiple Classification Problems

Authors : Ali Vasfi Ağlarcı, Cengiz Bal

Pages : 22-39

Doi:10.21180/iibfdkastamonu.1561910

View : 54 | Download : 28

Publication Date : 2025-06-30

Article Type : Research Paper

Abstract :The purpose of this research is to compare the performance metrics used in multiple classification problems in machine learning. For this purpose, simulation study was carried out under different scenarios by using 4 different classification methods and the performance metrics obtained were compared in this direction. While comparing the performance metrics in the study, the data to be used for classification purposes were derived under different scenarios, taking into account the effect of 4 factors. 90 different scenarios were created by considering the number of 3 different categories of the response variable, 5 different sample sizes, 3 different correlation structures, and the balanced and unbalanced distribution of the response variable. Accuray, Kappa and CramerV metrics used in multiple classification problems were used as performance measures. Changes in performance metrics in the determined scenarios are summarized in tables and compared. As a result of the comparisons made with the simulation study, it has been seen that Kappa performance measure is a more accurate performance metric than the other two metrics in multi-class classification problems, and the method gives more reliable information about the classification success.
Keywords : Sınıflandırma başarısı, sınıflandırma performansı, makine öğrenimi, simülasyon, performans ölçümleri

ORIGINAL ARTICLE URL