Sharp Trade-Offs in High-Dimensional Inference via 2-Level SLOPE

Among techniques for high-dimensional linear regression, Sorted L-One Penalized Estimation (SLOPE) generalizes the LASSO via an adaptive l1 regularization that applies heavier penalties to larger coefficients in the model. To achieve such adaptivity, SLOPE requires the specification of a complex hierarchy of penalties, i.e., a monotone penalty sequence in Rp, in contrast to a single penalty scalar for LASSO. Tuning this sequence when p is large poses a challenge, as brute force search over a grid of values is computationally prohibitive. In this work, we study the 2-level SLOPE, an important subclass of SLOPE, with only three hyperparameters. We demonstrate both empirically and analytically that 2-level SLOPE not only preserves the advantages of general SLOPE -- such as improved mean squared error and overcoming the Donoho-Tanner power limit -- but also exhibits computational benefits by reducing the penalty hyperparameter space. In particular, we prove that 2-level SLOPE admits a sharp, theoretically tight characterization of the trade-off between true positive proportion (TPP) and false discovery proportion (FDP), contrasting with general SLOPE where only upper and lower bounds are known. Empirical evaluations further underscore the effectiveness of 2-level SLOPE in settings where predictors exhibit high correlation, when the noise is large, or when the underlying signal is not sparse. Our results suggest that 2-level SLOPE offers a robust, scalable alternative to both LASSO and general SLOPE, making it particularly suited for practical high-dimensional data analysis.