# Anderson acceleration of coordinate descent

arXiv: https://arxiv.org/abs/2011.10065 · HTML: https://mlsanity.com/paper/2011.10065/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Nov 2020.

## Built on by (most cited first, 6 of 6)

- [Beyond L1: Faster and Better Sparse Models with skglm](https://mlsanity.com/paper/2204.07826/) (2022)
- [Anderson Acceleration For Bioinformatics-Based Machine Learning](https://mlsanity.com/paper/2302.00347/) (2023)
- [Improved Convergence Factor of Windowed Anderson Acceleration for Symmetric Fixed-Point Iterations](https://mlsanity.com/paper/2311.02490/) (2023)
- [Iterative Regularization with k-support Norm: An Important Complement to Sparse Recovery](https://mlsanity.com/paper/2401.05394/) (2023)
- [Anderson Acceleration in Nonsmooth Problems: Local Convergence via Active Manifold Identification](https://mlsanity.com/paper/2410.09420/) (2024)
- [Simpler Methods Work Better for L1 Penalized Logistic Models and Large Datasets](https://mlsanity.com/paper/2609.23995/) (2026)

## Abstract (arXiv)

Acceleration of first order methods is mainly obtained via inertial techniques `a la Nesterov, or via nonlinear extrapolation. The latter has known a recent surge of interest, with successful applications to gradient and proximal gradient techniques. On multiple Machine Learning problems, coordinate descent achieves performance significantly superior to full-gradient methods. Speeding up coordinate descent in practice is not easy: inertially accelerated versions of coordinate descent are theoretically accelerated, but might not always lead to practical speed-ups. We propose an accelerated version of coordinate descent using extrapolation, showing considerable speed up in practice, compared to inertial accelerated coordinate descent and extrapolated (proximal) gradient descent. Experiments on least squares, Lasso, elastic net and logistic regression validate the approach.