# Nash: Neural Adaptive Shrinkage for Structured High-Dimensional Regression

arXiv: https://arxiv.org/abs/2505.11143 · HTML: https://mlsanity.com/paper/2505.11143/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 46 references that are in the map.
- First posted on arXiv 16 May 2025.

## Builds on (most cited first, 14 of 14)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Noise2Self: Blind Denoising by Self-Supervision](https://mlsanity.com/paper/1901.11365/) (2019)
- [LassoNet: A Neural Network with Feature Sparsity](https://mlsanity.com/paper/1907.12207/) (2019)
- [Hybrid Models with Deep and Invertible Features](https://mlsanity.com/paper/1902.02767/) (2019)
- [Proximal Algorithms in Statistics and Machine Learning](https://mlsanity.com/paper/1502.03175/) (2015)
- [L0Learn: A Scalable Package for Sparse Learning using L0 Regularization](https://mlsanity.com/paper/2202.04820/) (2022)
- [Empirical Bayes Matrix Factorization](https://mlsanity.com/paper/1802.06931/) (2018)
- [Graph Mixture Density Networks](https://mlsanity.com/paper/2012.03085/) (2020)
- [A flexible empirical Bayes approach to multiple linear regression and connections with penalized regression](https://mlsanity.com/paper/2208.10910/) (2022)
- [A fast algorithm for maximum likelihood estimation of mixture proportions using sequential quadratic programming](https://mlsanity.com/paper/1806.01412/) (2018)
- [Sparse Regression with Multi-type Regularized Feature Modeling](https://mlsanity.com/paper/1810.03136/) (2018)
- [ebnm: An R Package for Solving the Empirical Bayes Normal Means Problem Using a Variety of Prior Families](https://mlsanity.com/paper/2110.00152/) (2021)

## Abstract (arXiv)

Sparse linear regression is a fundamental tool in data analysis. However, traditional approaches often fall short when covariates exhibit structure or arise from heterogeneous sources. In biomedical applications, covariates may stem from distinct modalities or be structured according to an underlying graph. We introduce textitNeural Adaptive Shrinkage (Nash), a unified framework that integrates covariate-specific side information into sparse regression via neural networks. Nash adaptively modulates penalties on a per-covariate basis, learning to tailor regularization without cross-validation. We use a textitsplit variational empirical Bayes algorithm that decouples prior learning from posterior inference, reducing the M-step from O(p)  neural-network passes per sweep to a single batched pass, a textit74 to 106x wall-clock speedup over previously proposed coordinate ascent CAVI for p between 102 and 104. Experiments on real data demonstrate that Nash improves accuracy and adaptability over existing methods.