Effect on model performance of regularization methods

Authors : Cafer BUDAK, Vasfiye MENÇİK, Mehmet Emin ASKER

Pages : 757-765

Doi:10.24012/dumf.1051352

View : 20 | Download : 16

Publication Date : 2021-12-31

Article Type : Research Paper

Abstract :Artificial Neural Networks with numerous parameters are tremendously powerful machine learning systems. Nonetheless, overfitting is a crucial problem in such networks. Maximizing the model accuracy and minimizing the amount of loss is significant in reducing in-class differences and maintaining sensitivity to these differences. In this study, the effects of overfitting for different model architectures with the Wine dataset were investigated by Dropout, AlfaDropout, GausianDropout, Batch normalization, Layer normalization, Activity normalization, L1 and L2 regularization methods and the change in loss function the combination with these methods. Combinations that performed well were examined on different datasets using the same model. The binary cross-entropy loss function was used as a performance measurement metric. According to the results, the Layer and Activity regularization combination showed better training and testing performance compared to other combinations.
Keywords : Overfitting, Machine learning, Regularization