# Proximal Gradient methods with Adaptive Subspace Sampling

arXiv: https://arxiv.org/abs/2004.13356 · HTML: https://mlsanity.com/paper/2004.13356/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Apr 2020.

## Built on by (most cited first, 2 of 2)

- [Global optimization using random embeddings](https://mlsanity.com/paper/2107.12102/) (2021)
- [Optimal low-rank stochastic gradient estimation for LLM training](https://mlsanity.com/paper/2603.20632/) (2026)

## Abstract (arXiv)

Many applications in machine learning or signal processing involve nonsmooth optimization problems. This nonsmoothness brings a low-dimensional structure to the optimal solutions. In this paper, we propose a randomized proximal gradient method harnessing this underlying structure. We introduce two key components: i) a random subspace proximal gradient algorithm; ii) an identification-based sampling of the subspaces. Their interplay brings a significant performance improvement on typical learning problems in terms of dimensions explored.