mlsanityData to 7 Oct 2026

Compositional federated learning: Applications in distributionally robust averaging and meta learning

arXiv 2106.11264 ↗First posted 21 Jun 2021cs.LG cs.DC math.OCcited by 12

Not on the map. Why?

Abstract (arXiv)

In the paper, we propose an effective and efficient Compositional Federated Learning (ComFedL) algorithm for solving a new compositional Federated Learning (FL) framework, which frequently appears in many data mining and machine learning problems with a hierarchical structure such as distributionally robust FL and model-agnostic meta learning (MAML). Moreover, we study the convergence analysis of our ComFedL algorithm under some mild conditions, and prove that it achieves a convergence rate of O(frac1√T), where T denotes the number of iteration. To the best of our knowledge, our new Compositional FL framework is the first work to bridge federated learning with composition stochastic optimization. In particular, we first transform the distributionally robust FL (i.e., a minimax optimization problem) into a simple composition optimization problem by using KL divergence regularization. At the same time, we also first transform the distribution-agnostic MAML problem (i.e., a minimax optimization problem) into a simple yet effective composition optimization problem. Finally, we apply two popular machine learning tasks, i.e., distributionally robust FL and MAML to demonstrate the effectiveness of our algorithm.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 46 of its 56 references that are in the map. First posted on arXiv 21 Jun 2021.

Builds on

46 of 56 references.

References · 46

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  4. 2019Advances and Open Problems in Federated Learningcited by 3,021
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  6. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  7. 2018Federated Learning with Non-IID Datacited by 1,099
  8. 2020Adaptive Federated Optimizationcited by 1,036
  9. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  10. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  11. 2016Learning to learn by gradient descent by gradient descentcited by 616
  12. 2020Personalized Federated Learning with Moreau Envelopescited by 612
34 more
  1. 2019Agnostic Federated Learningcited by 451
  2. 2019Fair Resource Allocation in Federated Learningcited by 389
  3. 2020Adaptive Personalized Federated Learningcited by 353
  4. 2020Personalized Federated Learning: A Meta-Learning Approachcited by 309
  5. 2018Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learningcited by 277
  6. 2019Tighter Theory for Local SGD on Identical and Heterogeneous Datacited by 266
  7. 2019Asynchronous Federated Optimizationcited by 264
  8. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 264
  9. 2019SCAFFOLD: Stochastic Controlled Averaging for Federated Learningcited by 174
  10. 2018Federated Meta-Learning with Fast Convergence and Efficient Communicationcited by 154
  11. 2020Mime: Mimicking Centralized Stochastic Algorithms in Federated Learningcited by 152
  12. 2020Is Local SGD Better than Minibatch SGD?cited by 151
  13. 2020FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Datacited by 143
  14. 2019Online Meta-Learningcited by 137
  15. 2019On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithmscited by 132
  16. 2019SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentumcited by 121
  17. 2019Local SGD with Periodic Averaging: Tighter Analysis and Adaptive Synchronizationcited by 106
  18. 2021Distributionally Robust Federated Averagingcited by 104
  19. 2020Robust Federated Learning: The Case of Affine Distribution Shiftscited by 102
  20. 2014Stochastic Compositional Gradient Descent: Algorithms for Minimizing Compositions of Expected-Value Functionscited by 93
  21. 2019Variance Reduced Local SGD with Lower Communication Complexitycited by 80
  22. 2019Provable Guarantees for Gradient-Based Meta-Learningcited by 67
  23. 2019ES-MAML: Simple Hessian-Free Meta Learningcited by 57
  24. 2018A Single Time-Scale Stochastic Approximation Method for Nested Stochastic Optimizationcited by 53
  25. 2020Solving Stochastic Compositional Optimization is Nearly as Easy as Solving Stochastic Optimizationcited by 51
  26. 2016Accelerating Stochastic Composition Optimizationcited by 46
  27. 2021FLOP: Federated Learning on Medical Datasets using Partial Networkscited by 42
  28. 2019Multi-Level Composite Stochastic Optimization via Nested Variance Reductioncited by 28
  29. 2020Theoretical Convergence of Multi-Step Model-Agnostic Meta-Learningcited by 28
  30. 2020FedCluster: Boosting the Convergence of Federated Learning via Cluster-Cyclingcited by 26
  31. 2017Accelerated Method for Stochastic Composition Optimization with Nonsmooth Regularizationcited by 13
  32. 2020Compositional ADAM: An Adaptive Compositional Solvercited by 8
  33. 2018Improved Sample Complexity for Stochastic Compositional Variance Reduced Gradientcited by 6
  34. 2020Task-Robust Model-Agnostic Meta-Learningcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 4; 2024: 1; 2025: 1; 2026: 1.

12 papers.

  1. 2022FedNest: Federated Bilevel, Minimax, and Compositional Optimizationcited by 60
  2. 2022Communication-Efficient Robust Federated Learning with Noisy Labelscited by 11
  3. 2023Federated Compositional Deep AUC Maximizationcited by 8
  4. 2022DR-DSGD: A Distributionally Robust Decentralized Learning Algorithm over Graphscited by 5
  5. 2022FedGRec: Federated Graph Recommender System with Lazy Update of Latent Embeddingscited by 4
  6. 2023Federated Conditional Stochastic Optimizationcited by 4
  7. 2022Faster Adaptive Momentum-Based Federated Methods for Distributed Composition Optimizationcited by 2
  8. 2023FedDRO: Federated Compositional Optimization for Distributionally Robust Learningcited by 1
  9. 2024FDR-SVM: A Federated Distributionally Robust Support Vector Machine via a Mixture of Wasserstein Balls Ambiguity Setcited by 1
  10. 2023Federated Multi-Sequence Stochastic Approximation with Local Hypergradient Estimationcited by 0
  11. 2025Semi-overlapping Multi-bandit Best Arm Identification for Sequential Support Network Learningcited by 0
  12. 2026Federated Compositional Muon Optimizer for Matrix-Wise Modelscited by 0