mlsanityData to 7 Oct 2026

AIDE: Fast and Communication Efficient Distributed Optimization

arXiv 1608.06879 ↗First posted 24 Aug 2016math.OC cs.LG stat.MLcited by 42

Not on the map. Why?

Abstract (arXiv)

In this paper, we present two new communication-efficient methods for distributed minimization of an average of functions. The first algorithm is an inexact variant of the DANE algorithm that allows any local algorithm to return an approximate solution to a local subproblem. We show that such a strategy does not affect the theoretical guarantees of DANE significantly. In fact, our approach can be viewed as a robustification strategy since the method is substantially better behaved than DANE on data partition arising in practice. It is well known that DANE algorithm does not match the communication complexity lower bounds. To bridge this gap, we propose an accelerated variant of the first method, called AIDE, that not only matches the communication lower bounds but can also be implemented using a purely first-order oracle. Our empirical results show that AIDE is superior to other communication efficient algorithms in settings that naturally arise in machine learning applications.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Aug 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 15; 2022: 10; 2023: 4; 2024: 7; 2025: 5; 2026: 1.

42 papers.

Sort:

  1. 2021FedNL: Making Newton-Type Methods Applicable to Federated Learningcited by 50
  2. 2021A Bayesian Federated Learning Framework with Online Laplace Approximationcited by 41
  3. 2021Communication Efficiency in Federated Learning: Achievements and Challengescited by 40
  4. 2021The Min-Max Complexity of Distributed Stochastic Convex Optimization with Intermittent Communicationcited by 30
  5. 2021Bias-Variance Reduced Local SGD for Less Heterogeneous Federated Learningcited by 30
  6. 2021Distributed Second Order Methods with Fast Rates and Compressed Communicationcited by 25
  7. 2022AdaBest: Minimizing Client Drift in Federated Learning via Adaptive Bias Estimationcited by 21
  8. 2021Acceleration in Distributed Optimization under Similaritycited by 17
  9. 2021Asynchronous Federated Learning for Sensor Data with Concept Driftcited by 14
  10. 2021Basis Matters: Better Communication-Efficient Second Order Methods for Federated Learningcited by 14
  11. 2022SHED: A Newton-type algorithm for federated learning based on incremental Hessian eigenvector sharingcited by 11
  12. 2022SAFARI: Sparsity enabled Federated Learning with Limited and Unreliable Communicationscited by 11
30 more
  1. 2022Distributed Newton-Type Methods with Communication Compression and Bernoulli Aggregationcited by 11
  2. 2023Stochastic Distributed Optimization under Average Second-order Similarity: Algorithms and Analysiscited by 11
  3. 2021On Second-order Optimization Methods for Federated Learningcited by 9
  4. 2022Optimal Gradient Sliding and its Application to Distributed Optimization Under Similaritycited by 9
  5. 2023Federated Empirical Risk Minimization via Second-Order Methodcited by 9
  6. 2021Distributed Saddle-Point Problems Under Similaritycited by 8
  7. 2021Linear Speedup in Personalized Collaborative Learningcited by 8
  8. 2022Compression and Data Similarity: Combination of Two Techniques for Communication-Efficient Solving of Distributed Variational Inequalitiescited by 8
  9. 2021A Stochastic Newton Algorithm for Distributed Convex Optimizationcited by 6
  10. 2024Stabilized Proximal-Point Methods for Federated Optimizationcited by 6
  11. 2022FedSSO: A Federated Server-Side Second-Order Optimization Algorithmcited by 3
  12. 2023Memory-Constrained Algorithms for Convex Optimization via Recursive Cutting-Planescited by 3
  13. 2021Delayed Projection Techniques for Linearly Constrained Problems: Convergence Rates, Acceleration, and Applicationscited by 2
  14. 2021CDMA: A Practical Cross-Device Federated Learning Algorithm for General Minimax Problemscited by 2
  15. 2022Escaping Saddle Points with Bias-Variance Reduced Local Perturbed SGD for Communication Efficient Nonconvex Distributed Learningcited by 2
  16. 2025Convergence of Agnostic Federated Averagingcited by 2
  17. 2022Communication-Efficient {Federated} Learning Using Censored Heavy Ball Descentcited by 1
  18. 2023Exploring the Robustness of Decentralized Training for Large Language Modelscited by 1
  19. 2024Optimal Data Splitting in Distributed Optimization for Machine Learningcited by 1
  20. 2024Gradient Descent is Pareto-Optimal in the Oracle Complexity and Memory Tradeoff for Feasibility Problemscited by 1
  21. 2024Newton Meets Marchenko-Pastur: Massively Parallel Second-Order Optimization with Hessian Sketching and Debiasingcited by 1
  22. 2024GP-FL: Model-Based Hessian Estimation for Second-Order Over-the-Air Federated Learningcited by 1
  23. 2022Better Methods and Theory for Federated Learning: Compression, Client Selection and Heterogeneitycited by 0
  24. 2024Flattened one-bit stochastic gradient descent: compressed distributed optimization with controlled variancecited by 0
  25. 2024Federated X-armed Bandit with Flexible Personalisationcited by 0
  26. 2025FedOSAA: Improving Federated Learning with One-Step Anderson Accelerationcited by 0
  27. 2025Accelerated Training of Federated Learning via Second-Order Methodscited by 0
  28. 2025Mitigating Persistent Client Dropout in Asynchronous Decentralized Federated Learningcited by 0
  29. 2025CoCoL: A Communication Efficient Decentralized Collaborative Method for Multi-Robot Systemscited by 0
  30. 2026FedeRage: Provably Convergent Agnostic Federated Learning under General Client Driftcited by 0