Geometry of Model Pattern Recovery by Penalized and Thresholded Estimators
Patrick Tardivel (University of Burgundy)
LASSO, SLOPE, OSCAR, Fused LASSO, Clustered LASSO, generalized LASSO… are popular penalized estimators for which the penalty term is a polyhedral gauge. This presentation focuses on the model pattern recovery of β; namely recovering the subdifferential of a polyhedral gauge at β, where β is an unknown parameter of regression coefficients. For LASSO, when the penalty term is the ℓ1 norm, the model pattern of β only depends on the sign of β and sign recovery via LASSO estimator is actually a well known topic in the literature. Furthermore, this presentation shows that the notion of model pattern recovery is relevant for many examples of polyhedral gauge penalty. Specifically, we introduce the “path condition”: a necessary condition for model pattern recovery, via a penalized least squares estimator, with a probability larger than 1/2. One may relax this later condition using “thresholded” penalized least squares estimators; a new class of estimators generalizing thresholded LASSO. Indeed, we show that the “accessibility condition”, a condition weaker than the “path condition”, is asymptotically sufficient for model pattern recovery. It is well known that penalized estimators can be not uniquely defined and,
actually, the theory of model pattern recovery is closely related to the important issue of uniqueness. In this presentation we also introduce a necessary and sufficient condition for the uniform uniqueness of penalized least squares estimators.