# Faster SGD Using Sketched Conditioning

arXiv: https://arxiv.org/abs/1506.02649 · HTML: https://mlsanity.com/paper/1506.02649/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jun 2015.

## Built on by (most cited first, 3 of 3)

- [MARS-M: When Variance Reduction Meets Matrices](https://mlsanity.com/paper/2510.21800/) (2025)
- [ISAAC Newton: Input-based Approximate Curvature for Newton's Method](https://mlsanity.com/paper/2305.00604/) (2023)
- [Towards a Principled Muon under μmathsfP: Ensuring Spectral Conditions throughout Training](https://mlsanity.com/paper/2601.01306/) (2026)

## Abstract (arXiv)

We propose a novel method for speeding up stochastic optimization algorithms via sketching methods, which recently became a powerful tool for accelerating algorithms for numerical linear algebra. We revisit the method of conditioning for accelerating first-order methods and suggest the use of sketching methods for constructing a cheap conditioner that attains a significant speedup with respect to the Stochastic Gradient Descent (SGD) algorithm. While our theoretical guarantees assume convexity, we discuss the applicability of our method to deep neural networks, and experimentally demonstrate its merits.