mlsanityData to 7 Oct 2026

On the Convergence of Momentum-Based Algorithms for Federated Bilevel Optimization Problems

arXiv 2204.13299 ↗First posted 28 Apr 2022cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

In this paper, we studied the federated bilevel optimization problem, which has widespread applications in machine learning. In particular, we developed two momentum-based algorithms for optimizing this kind of problem and established the convergence rate of our two algorithms, providing the sample and communication complexities. Importantly, to the best of our knowledge, our convergence rate is the first one achieving the linear speedup with respect to the number of devices for federated bilevel optimization algorithms. At last, our extensive experimental results confirm the effectiveness of our two algorithms.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 18 of its 31 references that are in the map. First posted on arXiv 28 Apr 2022.

Builds on

18 of 31 references.

References · 18

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  2. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  3. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  4. 2018Local SGD Converges Fast and Communicates Littlecited by 595
  5. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
  6. 2019Momentum-Based Variance Reduction in Non-Convex SGDcited by 302
  7. 2018SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimatorcited by 283
  8. 2018Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learningcited by 277
  9. 2017Forward and Reverse Gradient-Based Hyperparameter Optimizationcited by 232
  10. 2018Approximation Methods for Bilevel Programmingcited by 231
  11. 2020Bilevel Optimization: Convergence Analysis and Enhanced Designcited by 219
  12. 2019On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimizationcited by 178
6 more
  1. 2020Federated Learning with Compression: Unified Analysis and Sharp Guaranteescited by 159
  2. 2021Provably Faster Algorithms for Bilevel Optimizationcited by 106
  3. 2021STEM: A Stochastic Two-Sided Momentum Algorithm Achieving Near-Optimal Sample and Communication Complexities for Federated Learningcited by 43
  4. 2021BiAdam: Fast Adaptive Bilevel Optimization Methodscited by 33
  5. 2021Bias-Variance Reduced Local SGD for Less Heterogeneous Federated Learningcited by 30
  6. 2021Enhanced Bilevel Optimization via Bregman Distancecited by 28

Built on by

Not cited yet.