mlsanityData to 7 Oct 2026

FedDANE: A Federated Newton-Type Method

arXiv 2001.01920 ↗First posted 7 Jan 2020cs.LG stat.MLcited by 76

Not on the map. Why?

Abstract (arXiv)

Federated learning aims to jointly learn statistical models over massively distributed remote devices. In this work, we propose FedDANE, an optimization method that we adapt from DANE, a method for classical distributed optimization, to handle the practical constraints of federated learning. We provide convergence guarantees for this method when learning over both convex and non-convex functions. Despite encouraging theoretical results, we find that the method has underwhelming performance empirically. In particular, through empirical simulations on both synthetic and real-world datasets, FedDANE consistently underperforms baselines of FedAvg and FedProx in realistic federated settings. We identify low device participation and statistical device heterogeneity as two underlying causes of this underwhelming performance, and conclude by suggesting several directions of future work.

Key facts. Cited by 76 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Jan 2020.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 21; 2022: 18; 2023: 15; 2024: 17; 2025: 5.

76 papers.

Sort:

  1. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  2. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  3. 2021A Field Guide to Federated Optimizationcited by 312
  4. 2021Decentralized Federated Averagingcited by 114
  5. 2021Multi-Center Federated Learning: Clients Clustering for Better Personalizationcited by 112
  6. 2022Federated Learning with Label Distribution Skew via Logits Calibrationcited by 108
  7. 2021Distributionally Robust Federated Averagingcited by 104
  8. 2021FedCM: Federated Learning with Client-level Momentumcited by 77
  9. 2021On Large-Cohort Training for Federated Learningcited by 73
  10. 2022On the Importance and Applicability of Pre-Training for Federated Learningcited by 71
  11. 2022An Improved Analysis of Gradient Tracking for Decentralized Machine Learningcited by 68
  12. 2022Personalized Federated Learning via Variational Bayesian Inferencecited by 68
38 more
  1. 2022Communication-Efficient Stochastic Zeroth-Order Optimization for Federated Learningcited by 59
  2. 2023FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracycited by 49
  3. 2021Cluster-driven Graph Federated Learning over Multiple Domainscited by 43
  4. 2021Communication Efficiency in Federated Learning: Achievements and Challengescited by 40
  5. 2022On Convergence of FedProx: Local Dissimilarity Invariant Bounds, Non-smoothness and Beyondcited by 37
  6. 2022Multi-Level Branched Regularization for Federated Learningcited by 34
  7. 2024Relaxed Contrastive Learning for Federated Learningcited by 32
  8. 2021Local Adaptivity in Federated Learning: Convergence and Consistencycited by 31
  9. 2022On the Convergence of Clustered Federated Learningcited by 31
  10. 2022Personalized Federated Learning With Graphcited by 29
  11. 2023Understanding Generalization of Federated Learning via Stability: Heterogeneity Matterscited by 27
  12. 2022Communication-Efficient Federated Learning with Accelerated Client Gradientcited by 24
  13. 2021The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learningcited by 23
  14. 2022Gradient Masked Averaging for Federated Learningcited by 22
  15. 2022AdaBest: Minimizing Client Drift in Federated Learning via Adaptive Bias Estimationcited by 21
  16. 2022A General Theory for Federated Optimization with Asynchronous and Heterogeneous Clients Updatescited by 17
  17. 2021FedSAE: A Novel Self-Adaptive Federated Learning Framework in Heterogeneous Systemscited by 16
  18. 2021Differential Privacy Meets Federated Learning under Communication Constraintscited by 15
  19. 2021On Bridging Generic and Personalized Federated Learning for Image Classificationcited by 15
  20. 2022FL Games: A Federated Learning Framework for Distribution Shiftscited by 14
  21. 2021FedChain: Chained Algorithms for Near-Optimal Communication Cost in Federated Learningcited by 10
  22. 2021Resource-constrained Federated Edge Learning with Heterogeneous Data: Formulation and Analysiscited by 10
  23. 2021On Second-order Optimization Methods for Federated Learningcited by 9
  24. 2021Towards Efficient Synchronous Federated Training: A Survey on System Optimization Strategiescited by 9
  25. 2024Efficient Federated Learning against Heterogeneous and Non-stationary Client Unavailabilitycited by 9
  26. 2021A General Theory for Client Sampling in Federated Learningcited by 8
  27. 2023Making Batch Normalization Great in Federated Deep Learningcited by 8
  28. 2023Federated Zeroth-Order Optimization using Trajectory-Informed Surrogate Gradientscited by 8
  29. 2023Distributed Personalized Empirical Risk Minimizationcited by 6
  30. 2023Federated Learning via Variational Bayesian Inference: Personalization, Sparsity and Clusteringcited by 5
  31. 2023Blockchain-Based Federated Learning: Incentivizing Data Sharing and Penalizing Dishonest Behaviorcited by 5
  32. 2023A Practical Recipe for Federated Learning Under Statistical Heterogeneity Experimental Designcited by 5
  33. 2022Personalized Federated Learning with Exact Stochastic Gradient Descentcited by 4
  34. 2024Rendering Wireless Environments Useful for Gradient Estimators: A Zero-Order Stochastic Federated Learning Methodcited by 4
  35. 2024Federated Learning Client Pruning for Noisy Labelscited by 4
  36. 2022FedSSO: A Federated Server-Side Second-Order Optimization Algorithmcited by 3
  37. 2023FedBug: A Bottom-Up Gradual Unfreezing Framework for Federated Learningcited by 3
  38. 2023Communication-Efficient Heterogeneous Federated Learning with Generalized Heavy-Ball Momentumcited by 3

50 most cited of 76 All 76 (JSON)