De-Biasing The Lasso With Degrees-of-Freedom Adjustment

This paper studies schemes to de-bias the Lasso in a linear model y=Xβ+ε where the goal is to construct confidence intervals for a0Tβ in a direction a0, where X has iid N(0,Sigma) rows. We show that previously analyzed propositions to de-bias the Lasso require a modification in order to enjoy efficiency in a full range of sparsity. This modification takes the form of a degrees-of-freedom adjustment that accounts for the dimension of the model selected by Lasso. Let s0 be the true sparsity. If Sigma is known and the ideal score vector proportional to XSigma-1a0 is used, the unadjusted de-biasing schemes proposed previously enjoy efficiency if s0lll n2/3. However, if s0ggg n2/3, the unadjusted schemes cannot be efficient in certain a0: then it is necessary to modify existing procedures by a degrees-of-freedom adjustment. This modification grants asymptotic efficiency for any a0 when s0/p→ 0 and s0log(p/s0)/n → 0. If Sigma is unknown, efficiency is granted for general a0 when fracs0log pn+minBigfracs_Omegalog pn,frac|Sigma-1a0|1√log p|Sigma-1/2a0|2 √ nBig+fracmin(s_Omega,s0)log p√ n→0 where s_Omega=|Sigma-1a0|0, provided that the de-biased estimate is modified with the degrees-of-freedom adjustment. The dependence in s0,s_Omega and |Sigma-1a0|1 is optimal. Our estimated score vector provides a novel methodology to handle dense a0. Our analysis shows that the degrees-of-freedom adjustment is not needed when the initial bias in direction a0 is small, which is granted under stringent conditions on Sigma-1. The main proof argument is an interpolation path similar to that typically used to derive Slepian's lemma. It yields a new ℓ_∞ error bound for the Lasso which is of independent interest.