mlsanityData to 7 Oct 2026

Distributed Coordinate Descent Method for Learning with Big Data

arXiv 1310.2059 ↗First posted 8 Oct 2013stat.ML cs.DC cs.LGcited by 23

Not on the map. Why?

Abstract (arXiv)

In this paper we develop and analyze Hydra: HYbriD cooRdinAte descent method for solving loss minimization problems with big data. We initially partition the coordinates (features) and assign each partition to a different node of a cluster. At every iteration, each node picks a random subset of the coordinates from those it owns, independently from the other computers, and in parallel computes and applies updates to the selected coordinates based on a simple closed-form formula. We give bounds on the number of iterations sufficient to approximately solve the problem with high probability, and show how it depends on the data and on the partitioning. We perform numerical experiments with a LASSO instance described by a 3TB matrix.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Oct 2013.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 1; 2023: 5; 2024: 8; 2025: 3; 2026: 1.

23 papers.

Sort:

  1. 2021A Field Guide to Federated Optimizationcited by 312
  2. 2021Cross-Gradient Aggregation for Decentralized Learning from Non-IID datacited by 32
  3. 2024Accelerating Heterogeneous Federated Learning with Closed-form Classifierscited by 10
  4. 2023Federated Empirical Risk Minimization via Second-Order Methodcited by 9
  5. 2025Leveraging Coordinate Momentum in SignSGD and Muon: Memory-Optimized Zero-Ordercited by 9
  6. 2024FRUGAL: Memory-Efficient Optimization by Reducing State Overhead for Scalable Trainingcited by 7
  7. 2026FedBCD:Communication-Efficient Accelerated Block Coordinate Gradient Descent for Federated Learningcited by 7
  8. 2021Cross-Silo Federated Learning for Multi-Tier Networks with Vertical and Horizontal Data Partitioningcited by 6
  9. 2023Towards a Better Theoretical Understanding of Independent Subnetwork Trainingcited by 6
  10. 2021Stability and Generalization for Randomized Coordinate Descentcited by 4
  11. 2024Learning Scalable Model Soup on a Single GPU: An Efficient Subspace Training Strategycited by 3
  12. 2024High-Dimensional Distributed Sparse Classification with Scalable Communication-Efficient Global Updatescited by 3
11 more
  1. 2025Non-Euclidean Broximal Point Method: A Blueprint for Geometry-Aware Optimizationcited by 3
  2. 2022FLECS-CGD: A Federated Learning Second-Order Framework via Compression and Sketching with Compressed Gradient Differencescited by 2
  3. 2024A Mirror Descent-Based Algorithm for Corruption-Tolerant Distributed Gradient Descentcited by 2
  4. 2021Assisted Learning for Organizations with Limited Imbalanced Datacited by 1
  5. 2024Optimizing the Optimal Weighted Average: Efficient Distributed Sparse Classificationcited by 1
  6. 2025Federated Learning for Commercial Image Sourcescited by 1
  7. 2023Laplacian-based Semi-Supervised Learning in Multilayer Hypergraphs by Coordinate Descentcited by 0
  8. 2023Batches Stabilize the Minimum Norm Risk in High Dimensional Overparameterized Linear Regressioncited by 0
  9. 2023Scalable High-Dimensional Multivariate Linear Regression for Feature-Distributed Datacited by 0
  10. 2024Estimation Network Design framework for efficient distributed optimizationcited by 0
  11. 2024Beyond adaptive gradient: Fast-Controlled Minibatch Algorithm for large-scale optimizationcited by 0