# A Primer on Coordinate Descent Algorithms

arXiv: https://arxiv.org/abs/1610.00040 · HTML: https://mlsanity.com/paper/1610.00040/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Sep 2016.

## Built on by (most cited first, 17 of 17)

- [Toward Understanding Why Adam Converges Faster Than SGD for Transformers](https://mlsanity.com/paper/2306.00204/) (2023)
- [Order Matters: Agent-by-agent Policy Optimization](https://mlsanity.com/paper/2302.06205/) (2023)
- [Unlocking the Potential of Prompt-Tuning in Bridging Generalized and Personalized Federated Learning](https://mlsanity.com/paper/2310.18285/) (2023)
- [Differentially Private Coordinate Descent for Composite Empirical Risk Minimization](https://mlsanity.com/paper/2110.11688/) (2021)
- [High-Dimensional Private Empirical Risk Minimization by Greedy Coordinate Descent](https://mlsanity.com/paper/2207.01560/) (2022)
- [Convergence Rates of Training Deep Neural Networks via Alternating Minimization Methods](https://mlsanity.com/paper/2208.14318/) (2022)
- [Ky Fan Norms and Beyond: Dual Norms and Combinations for Matrix Optimization](https://mlsanity.com/paper/2512.09678/) (2025)
- [Cyclic Coordinate Dual Averaging with Extrapolation](https://mlsanity.com/paper/2102.13244/) (2021)
- [Multiview Graph Learning with Consensus Graph](https://mlsanity.com/paper/2401.13769/) (2024)
- [Scalable Kernel Inverse Optimization](https://mlsanity.com/paper/2410.23952/) (2024)
- [WarpAdam: A new Adam optimizer based on Meta-Learning approach](https://mlsanity.com/paper/2409.04244/) (2024)
- [Block-Structured Optimization for Subgraph Detection in Interdependent Networks](https://mlsanity.com/paper/2210.02702/) (2022)
- [The Landscape of Modern Machine Learning: A Review of Machine, Distributed and Federated Learning](https://mlsanity.com/paper/2312.03120/) (2023)
- [Generalizing Trilateration: Approximate Maximum Likelihood Estimator for Initial Orbit Determination in Low-Earth Orbit](https://mlsanity.com/paper/2407.15180/) (2024)
- [Differentially Private Random Block Coordinate Descent](https://mlsanity.com/paper/2412.17054/) (2024)
- [Solving Constrained Stochastic Shortest Path Problems with Scalarisation](https://mlsanity.com/paper/2508.17446/) (2025)
- [Student Copula Markov Networks](https://mlsanity.com/paper/2610.19135/) (2026)

## Abstract (arXiv)

This monograph presents a class of algorithms called coordinate descent algorithms for mathematicians, statisticians, and engineers outside the field of optimization. This particular class of algorithms has recently gained popularity due to their effectiveness in solving large-scale optimization problems in machine learning, compressed sensing, image processing, and computational statistics. Coordinate descent algorithms solve optimization problems by successively minimizing along each coordinate or coordinate hyperplane, which is ideal for parallelized and distributed computing. Avoiding detailed technicalities and proofs, this monograph gives relevant theory and examples for practitioners to effectively apply coordinate descent to modern problems in data science and engineering.