Robust Stepwise Algorithms for Linear Regression: A Comparative Study
Keywords:
Computational complexity, Forward Selection, Least angle regression, Linear regression, Robust prediction, Stepwise procedure, Winsorization.Abstract
For building a linear prediction model, Forward selection (FS) (Weisberg 1985) and Least Angle Regression (LARS) (Efron et al. 2004) are
two efficient stepwise procedures for sequencing the candidate predictors. Both the methods yield poor results when data contain outliers
and other contaminations. Khan et al. (2007a) and Khan et al. (2007b) proposed robust versions of LARS (RLARS) and FS (RFS),
respectively, which are computationally very suitable and scalable to large high-dimensional datasets. However, no comparison has been
made between RFS and RLARS. In this study, we compare these two stepwise algorithms. We conduct an extensive simulation study to
compare the number of correct covariates identified by these two algorithms in linear regression. We also apply these algorithms to
empirical data. Based on our simulation study and real-data application, the efficiency of RFS appears to be better than that of RLARS.
