Autotune: fast, accurate, and automatic tuning parameter selection for Lasso

Least absolute shrinkage and selection operator (Lasso), the popular variable selection engine for high-dimensional regression, is commonly tuned using cross-validation (CV). This is known to be slow and loses accuracy in high-dimension, low signal-to-noise ratio (SNR) settings. These issues are exacerbated for high-dimensional time series models such as the vector autoregression (VAR), where time series cross-validaiton (TSCV) is used for tuning. We propose textttautotune, a strategy for the Lasso to tune itself automatically by exploiting information contained in partial residuals computed during a single Lasso fit. The strategy can also be viewed as alternately estimating noise standard deviation and column space of relevant predictors. Numerical experiments on regression and VAR models show that textttautotune is faster than existing alternatives, and more accurate when the SNR is low. It also provides an accurate estimator of noise scale and diagnostic plots akin to screeplots for checking model sparsity. We demonstrate the benefit of textttautotune on several real data sets and develop an R package available on CRAN.