mlsanityData to 14 Oct 2026

Efficient Distributed Learning with Sparsity

arXiv 1605.07991 ↗First posted 25 May 2016stat.ML cs.LGcited by 38

Not on the map. Why?

Abstract (arXiv)

We propose a novel, efficient approach for distributed sparse learning in high-dimensions, where observations are randomly partitioned across machines. Computationally, at each round our method only requires the master machine to solve a shifted ell_1 regularized M-estimation problem, and other workers to compute the gradient. In respect of communication, the proposed approach provably matches the estimation error bound of centralized methods within constant rounds of communications (ignoring logarithmic factors). We conduct extensive experiments on both simulated and real world datasets, and demonstrate encouraging performances on high-dimensional regression and classification tasks.

Key facts. Cited by 38 papers in the map (data through 14 Oct 2026). First posted on arXiv 25 May 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 13; 2022: 9; 2023: 3; 2024: 7; 2025: 3; 2026: 3.

38 papers.

Sort:

  1. 2022Fine-tuning Language Models over Slow Networks using Activation Compression with Guaranteescited by 30
  2. 2021An Adaptive Stochastic Sequential Quadratic Programming with Differentiable Exact Augmented Lagrangianscited by 25
  3. 2021Federated Causal Inference in Heterogeneous Observational Datacited by 24
  4. 2021Targeting Underrepresented Populations in Precision Medicine: A Federated Transfer Learning Approachcited by 20
  5. 2021Distributed Learning Systems with First-order Methodscited by 17
  6. 2021Persia: An Open, Hybrid System Scaling Deep Learning-based Recommenders up to 100 Trillion Parameterscited by 17
  7. 2021BAGUA: Scaling up Distributed Learning with System Relaxationscited by 13
  8. 2022Fast Composite Optimization and Statistical Recovery in Federated Learningcited by 12
  9. 2025Subspace Networks: Scaling Decentralized Training with Communication-Efficient Model Parallelismcited by 12
  10. 2022A Communication-efficient Algorithm with Linear Convergence for Federated Minimax Learningcited by 9
  11. 2021Communication-Constrained Distributed Quantile Regression with Optimal Statistical Guaranteescited by 8
  12. 2025Shift is Good: Mismatched Data Mixing Improves Test Performancecited by 5
26 more
  1. 2021Distributed Bootstrap for Simultaneous Inference Under High Dimensionalitycited by 4
  2. 2022Fast and Robust Sparsity Learning over Networks: A Decentralized Surrogate Median Regression Approachcited by 4
  3. 2023Sharper Analysis for Minibatch Stochastic Proximal Point Methods: Stability, Smoothness, and Deviationcited by 4
  4. 2024CG-FedLLM: How to Compress Gradients in Federated Fune-tuning for Large Language Modelscited by 4
  5. 2021Distributed Sparse Normal Means Estimation with Sublinear Communicationcited by 3
  6. 2024High-Dimensional Distributed Sparse Classification with Scalable Communication-Efficient Global Updatescited by 3
  7. 2026AsyncMesh: Fully Asynchronous Optimization for Data and Pipeline Parallelismcited by 3
  8. 2021Distributed Sparse Feature Selection in Communication-Restricted Networkscited by 2
  9. 2024Distributed High-Dimensional Quantile Regression: Estimation Efficiency and Support Recoverycited by 2
  10. 2021Distributed Sparse Regression via Penalizationcited by 1
  11. 2022CEDAR: Communication Efficient Distributed Analysis for Regressionscited by 1
  12. 2022Majority Vote for Distributed Differentially Private Sign Selectioncited by 1
  13. 2023Distributed Semi-Supervised Sparse Statistical Inferencecited by 1
  14. 2024Optimizing the Optimal Weighted Average: Efficient Distributed Sparse Classificationcited by 1
  15. 2025Learning Pareto manifolds in high dimensions: How can regularization help?cited by 1
  16. 2026Learned Subspace Compression for Communication-Efficient Pipeline Parallelismcited by 1
  17. 2021Divide-and-conquer methods for big data analysiscited by 0
  18. 2021Communication-efficient Byzantine-robust distributed learning with statistical guaranteecited by 0
  19. 2022Distributed Sparse Multicategory Discriminant Analysiscited by 0
  20. 2022Recovery Guarantees for Distributed-OMPcited by 0
  21. 2022Distributed Estimation and Inference for Semi-parametric Binary Response Modelscited by 0
  22. 2023Communication-Efficient Distributed Estimation and Inference for Cox's Modelcited by 0
  23. 2024A Selective Review on Statistical Methods for Massive Data Computation: Distributed Computing, Subsampling, and Minibatch Techniquescited by 0
  24. 2024Using ARIMA to Predict the Expansion of Subscriber Data Consumptioncited by 0
  25. 2024Minimax and Communication-Efficient Distributed Best Subset Selection with Oracle Propertycited by 0
  26. 2026Byzantine-Robust Distributed Sparse Learning Revisitedcited by 0