Nash: Neural Adaptive Shrinkage for Structured High-Dimensional Regression

Sparse linear regression is a fundamental tool in data analysis. However, traditional approaches often fall short when covariates exhibit structure or arise from heterogeneous sources. In biomedical applications, covariates may stem from distinct modalities or be structured according to an underlying graph. We introduce textitNeural Adaptive Shrinkage (Nash), a unified framework that integrates covariate-specific side information into sparse regression via neural networks. Nash adaptively modulates penalties on a per-covariate basis, learning to tailor regularization without cross-validation. We use a textitsplit variational empirical Bayes algorithm that decouples prior learning from posterior inference, reducing the M-step from O(p) neural-network passes per sweep to a single batched pass, a textit74 to 106x wall-clock speedup over previously proposed coordinate ascent CAVI for p between 102 and 104. Experiments on real data demonstrate that Nash improves accuracy and adaptability over existing methods.