mlsanityData to 7 Oct 2026

PersA-FL: Personalized Asynchronous Federated Learning

arXiv 2210.01176 ↗First posted 3 Oct 2022cs.LG cs.AI cs.DCcited by 5

Not on the map. Why?

Abstract (arXiv)

We study the personalized federated learning problem under asynchronous updates. In this problem, each client seeks to obtain a personalized model that simultaneously outperforms local and global models. We consider two optimization-based frameworks for personalization: (i) Model-Agnostic Meta-Learning (MAML) and (ii) Moreau Envelope (ME). MAML involves learning a joint model adapted for each client through fine-tuning, whereas ME requires a bi-level optimization problem with implicit gradients to enforce personalization via regularized losses. We focus on improving the scalability of personalized federated learning by removing the synchronous communication assumption. Moreover, we extend the studied function class by removing boundedness assumptions on the gradient norm. Our main technical contribution is a unified proof for asynchronous federated learning with bounded staleness that we apply to MAML and ME personalization frameworks. For the smooth and non-convex functions class, we show the convergence of our method to a first-order stationary point. We illustrate the performance of our method and its tolerance to staleness through experiments for classification tasks over heterogeneous datasets.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 63 of its 82 references that are in the map. First posted on arXiv 3 Oct 2022.

Builds on

63 of 82 references.

References · 63

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2019Advances and Open Problems in Federated Learningcited by 3,021
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  5. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  6. 2019A Comprehensive Survey on Transfer Learningcited by 1,154
  7. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  8. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  9. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  10. 2018Local SGD Converges Fast and Communicates Littlecited by 595
  11. 2014Convex Optimization: Algorithms and Complexitycited by 569
  12. 2018Federated Learning for Mobile Keyboard Predictioncited by 565
38 more
  1. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  2. 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 528
  3. 2019Meta-Learning with Implicit Gradientscited by 523
  4. 2020Adaptive Personalized Federated Learningcited by 353
  5. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 321
  6. 2021A Field Guide to Federated Optimizationcited by 312
  7. 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 282
  8. 2018Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learningcited by 277
  9. 2019Tighter Theory for Local SGD on Identical and Heterogeneous Datacited by 266
  10. 2019Asynchronous Federated Optimizationcited by 264
  11. 2018Meta-Learning: A Surveycited by 254
  12. 2021Personalized Federated Learning using Hypernetworkscited by 210
  13. 2020Personalized Federated Learning with First Order Model Optimizationcited by 201
  14. 2015Federated Optimization:Distributed Optimization Beyond the Datacentercited by 193
  15. 2021Federated Learning with Buffered Asynchronous Aggregationcited by 188
  16. 2020Survey of Personalization Techniques for Federated Learningcited by 144
  17. 2019Online Meta-Learningcited by 137
  18. 2019On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithmscited by 132
  19. 2021The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregationcited by 127
  20. 2020Lower Bounds and Optimal Algorithms for Personalized Federated Learningcited by 122
  21. 2019Decentralized Deep Learning with Arbitrary Communication Compressioncited by 109
  22. 2022Sharper Convergence Guarantees for Asynchronous SGD for Distributed and Federated Learningcited by 76
  23. 2022Asynchronous SGD Beats Minibatch SGD Under Arbitrary Delayscited by 61
  24. 2020Straggler-Resilient Federated Learning: Leveraging the Interplay Between Statistical Accuracy and System Heterogeneitycited by 56
  25. 2021Convergence and Accuracy Trade-Offs in Federated Learning and Meta-Learningcited by 52
  26. 2021FedNL: Making Newton-Type Methods Applicable to Federated Learningcited by 50
  27. 2018A Tight Convergence Analysis for Stochastic Gradient Descent with Delayed Updatescited by 47
  28. 2019Semi-Cyclic Stochastic Gradient Descentcited by 40
  29. 2020Advances in Asynchronous Parallel and Distributed Optimizationcited by 40
  30. 2021Generalization of Model-Agnostic Meta-Learning Algorithms: Recurring and Unseen Taskscited by 36
  31. 2021Personalized Federated Learning: A Unified Framework and Universal Optimization Techniquescited by 34
  32. 2016Asynchrony begets Momentum, with an Application to Deep Learningcited by 30
  33. 2020Theoretical Convergence of Multi-Step Model-Agnostic Meta-Learningcited by 28
  34. 2014Differentially Private Distributed Optimizationcited by 23
  35. 2018Robust Asynchronous Stochastic Gradient-Push: Asymptotically Optimal and Network-Independent Performance for Strongly Convex Functionscited by 19
  36. 2020On the Convergence Theory of Debiased Model-Agnostic Meta-Reinforcement Learningcited by 19
  37. 2019Asynchronous Federated Learning with Differential Privacy for Edge Intelligencecited by 16
  38. 2021Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updatescited by 16

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 1; 2025: 1; 2026: 1.

5 papers.

  1. 2023Adaptive Federated Learning with Auto-Tuned Clientscited by 7
  2. 2024A Moreau Envelope Approach for LQR Meta-Policy Estimationcited by 2
  3. 2023On First-Order Meta-Reinforcement Learning with Moreau Envelopescited by 1
  4. 2025Federated Learning in Offline and Online EMG Decoding: A Privacy and Performance Perspectivecited by 0
  5. 2026Personalized Federated Reinforcement Learning via Model-Agnostic Meta-Learning: Convergence of Exact and Hessian-Free Meta-Policy Gradientscited by 0