On shrinkage and selection: ANOVA model

A. K. Md. Ehsanes Saleh, M. Arashi, M. Norouzirad, B M Goalm Kibria

Abstract


This paper considers the estimation of the parameters of an ANOVA model
when sparsity is suspected. Accordingly, we consider the least square estima-
tor (LSE), restricted LSE, preliminary test and Stein-type estimators, together
with three penalty estimators, namely, the ridge estimator, subset selection rules
(hard threshold estimator) and the LASSO (soft threshold estimator). We com-
pare and contrast the L2-risk of all the estimators with the lower bound of L2-
risk of LASSO in a family of diagonal projection scheme which is also the lower
bound of the exact L2-risk of LASSO. The result of this comparison is that nei-
ther LASSO nor the LSE, preliminary test, and Stein-type estimators outperform
each other uniformly. However, when the model is sparse, LASSO outperforms
all estimators except “ridge” estimator since both LASSO and ridge are L2-risk
equivalent under sparsity. We also nd that LASSO and the restricted LSE are
L2-risk equivalent and both outperform all estimators (except ridge) depending
on the dimension of sparsity. Finally, ridge estimator outperforms all estimators
uniformly. Our nding are based on L2-risk of estimators and lower bound of the
risk of LASSO together with tables of efficiency and graphical display of efficiency
and not based on simulation.

Keywords


ANOVA model, Dominance, Eciency, LASSO, PTE and Stein-type estimators, Penalty estimator, Risk function, Sparsity

Full Text:

PDF

Refbacks

  • There are currently no refbacks.


ISSN: 0256422X