Building a Robust Linear Model with Backward Elimination Procedure

Abstract

For building a linear prediction model, Backward Elimination (BE) is a computationally suitable stepwise procedure for sequencing the candidate predictors. This method yields poor results when data contain outliers and other contaminations. Robust model selection procedures, on the other hand, are not computationally efficient or scalable to large dimensions, because they require the fitting of a large number of submodels. Robust version of BE is proposed in this study, which is computationally very suitable and scalable to large high-dimensional data sets. Since BE can be expressed in terms of sample correlations, simple robustifications are obtained by replacing these correlations by their robust counterparts. A pairwise approach is used to construct the robust correlation matrix -- not only because of its computational advantages over the d-dimensional approach, but also because the pairwise approach is more consistent with the idea of step-by-step algorithms. The performance of the proposed robust method is much better than standard BE.