mlsanityData to 7 Oct 2026

Federated Learning on Adaptively Weighted Nodes by Bilevel Optimization

arXiv 2207.10751 ↗First posted 21 Jul 2022cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

We propose a federated learning method with weighted nodes in which the weights can be modified to optimize the model's performance on a separate validation set. The problem is formulated as a bilevel optimization where the inner problem is a federated learning problem with weighted nodes and the outer problem focuses on optimizing the weights based on the validation performance of the model returned from the inner problem. A communication-efficient federated optimization algorithm is designed to solve this bilevel optimization problem. Under an error-bound assumption, we analyze the generalization performance of the output model and identify scenarios when our method is in theory superior to training a model only locally and to federated learning with static and evenly distributed weights.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 30 of its 49 references that are in the map. First posted on arXiv 21 Jul 2022.

Builds on

30 of 49 references.

References · 30

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  5. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  6. 2017Federated Multi-Task Learningcited by 669
  7. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  8. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  9. 2019Meta-Learning with Implicit Gradientscited by 523
  10. 2021Towards Personalized Federated Learningcited by 462
  11. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
  12. 2018Bilevel Programming for Hyperparameter Optimization and Meta-Learningcited by 403
18 more
  1. 2017A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasetscited by 374
  2. 2020Adaptive Personalized Federated Learningcited by 353
  3. 2020Personalized Federated Learning: A Meta-Learning Approachcited by 309
  4. 2018Approximation Methods for Bilevel Programmingcited by 231
  5. 2018Truncated Back-propagation for Bilevel Optimizationcited by 187
  6. 2020A Two-Timescale Framework for Bilevel Optimization: Complexity Analysis and Application to Actor-Criticcited by 141
  7. 2020Local SGD: Unified Theory and New Efficient Methodscited by 88
  8. 2020Federated Accelerated Stochastic Gradient Descentcited by 85
  9. 2019Variance Reduced Local SGD with Lower Communication Complexitycited by 80
  10. 2020Fast-Convergent Federated Learning with Adaptive Weightingcited by 67
  11. 2022FedNest: Federated Bilevel, Minimax, and Compositional Optimizationcited by 60
  12. 2021Weighted Training for Cross-Task Learningcited by 22
  13. 2022Local Stochastic Bilevel Optimization with Momentum-Based Variance Reductioncited by 20
  14. 2021FedPAGE: A Fast Local Stochastic Gradient Method for Communication-Efficient Federated Learningcited by 19
  15. 2015RSG: Beating Subgradient Method without Smoothness and Strong Convexitycited by 15
  16. 2016Inexact Proximal Gradient Methods for Non-convex and Non-smooth Optimizationcited by 5
  17. 2017Faster Subgradient Methods for Functions with H"olderian Growthcited by 5
  18. 2014Linear Convergence of Variance-Reduced Stochastic Gradient without Strong Convexitycited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 2; 2024: 1; 2025: 0; 2026: 1.

6 papers.

  1. 2023SimFBO: Towards Simple, Flexible and Communication-efficient Federated Bilevel Learningcited by 14
  2. 2023Communication-Efficient Federated Hypergradient Computation via Aggregated Iterative Differentiationcited by 13
  3. 2022FeDXL: Provable Federated Learning for Deep X-Risk Optimizationcited by 10
  4. 2022Alternating Implicit Projected SGD and Its Efficient Variants for Equality-constrained Bilevel Optimizationcited by 10
  5. 2024Federated Learning Can Find Friends That Are Advantageouscited by 1
  6. 2026Provably Communication-Efficient and Privacy-Preserving Federated Graph Neural Networkscited by 0