Performance Comparison of Least Squares, Ridge, Lasso and Principal Component Regression for Addressing Multicollinearity in Regression Analysis

Authors : Semih Ergişi, Beyza Doğanay, Yasemin Yavuz

Pages : 59-72

View : 147 | Download : 105

Publication Date : 2024-12-31

Article Type : Research Paper

Abstract :The purpose of this research was to evaluate the predictive accuracy of various regression methods in the context of multiple linear regression when multicollinearity invalidates the underlying assumptions of the least squares method. These methods included least squares (LS), ridge regression (RR), lasso regression (LR), and principal component regression (PCR). For this aim, the dataset including 6 variables simulated from normal with different sample of size from range of 50 to 1000. The performance was assessed using mean square error (MSE) and R square value. Despite the existence of multicollienarity among independent variables, research findings showed that LS method had the smallest MSE in the training dataset but RR had the smallest mse in the test dataset. When the sample size increases, the mse values increase for each methods in the training set but decrease in the test set. They are closer to each other. In terms of R square values, all methods showed similar performance both training and test data set.
Keywords : Lasso regresyon, Temel bileşenler, Çoklu bağlantı, En küçük kareler regresyon