mlsanityData to 7 Oct 2026

Personalized Federated Learning through Local Memorization

arXiv 2111.09360 ↗First posted 17 Nov 2021cs.LG stat.MLcited by 57

Not on the map. Why?

Abstract (arXiv)

Federated learning allows clients to collaboratively learn statistical models while keeping their data local. Federated learning was originally used to train a unique global model to be served to all clients, but this approach might be sub-optimal when clients' local data distributions are heterogeneous. In order to tackle this limitation, recent personalized federated learning methods train a separate model for each client while still leveraging the knowledge available at other clients. In this work, we exploit the ability of deep neural networks to extract high quality vectorial representations (embeddings) from non-tabular data, e.g., images and text, to propose a personalization mechanism based on local memorization. Personalization is obtained by interpolating a collectively trained global model with a local k-nearest neighbors (kNN) model based on the shared representation provided by the global model. We provide generalization bounds for the proposed approach in the case of binary classification, and we show on a suite of federated datasets that this approach achieves significantly higher accuracy and fairness than state-of-the-art methods.

Key facts. Cited by 57 papers in the map (data through 7 Oct 2026). Builds on 50 of its 69 references that are in the map. First posted on arXiv 17 Nov 2021.

Builds on

50 of 69 references.

References · 50

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  3. 2019Advances and Open Problems in Federated Learningcited by 3,021
  4. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  5. 2017Billion-scale similarity search with GPUscited by 2,477
  6. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  7. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  8. 2020Adaptive Federated Optimizationcited by 1,036
  9. 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 853
  10. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
  11. 2018LEAF: A Benchmark for Federated Settingscited by 780
  12. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
38 more
  1. 2017Federated Multi-Task Learningcited by 669
  2. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  3. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  4. 2020Federated Learning with Matched Averagingcited by 564
  5. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  6. 2019Federated Learning with Personalization Layerscited by 533
  7. 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 528
  8. 2020An Efficient Framework for Clustered Federated Learningcited by 505
  9. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
  10. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
  11. 2019Agnostic Federated Learningcited by 451
  12. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 380
  13. 2020Adaptive Personalized Federated Learningcited by 353
  14. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 349
  15. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 334
  16. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 322
  17. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 321
  18. 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 282
  19. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 264
  20. 2021Personalized Federated Learning using Hypernetworkscited by 210
  21. 2020Nearest Neighbor Machine Translationcited by 209
  22. 2020Federated Learning of a Mixture of Global and Local Modelscited by 208
  23. 2018Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learningcited by 203
  24. 2020Personalized Federated Learning with First Order Model Optimizationcited by 201
  25. 2019SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learningcited by 194
  26. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  27. 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 189
  28. 2019Adaptive Gradient-Based Meta-Learning Methodscited by 183
  29. 2020Salvaging Federated Learning by Local Adaptationcited by 142
  30. 2020Lower Bounds and Optimal Algorithms for Personalized Federated Learningcited by 122
  31. 2015Inferring Algorithmic Patterns with Stack-Augmented Recurrent Netscited by 76
  32. 2021Personalized Federated Learning with Gaussian Processescited by 76
  33. 2016Decentralized Collaborative Learning of Personalized Models over Networkscited by 73
  34. 2019Variational Federated Multi-Task Learningcited by 69
  35. 2015Learning to Transduce with Unbounded Memorycited by 54
  36. 2019Fully Decentralized Joint Learning of Personalized Models and Collaboration Graphscited by 34
  37. 2019An Investigation Into On-device Personalization of End-to-end Automatic Speech Recognition Modelscited by 25
  38. 2018A Simple Cache Model for Image Recognitioncited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 20; 2024: 19; 2025: 9; 2026: 3.

57 papers.

Sort:

By area:

  1. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
  2. 2023Personalized Federated Learning with Feature Alignment and Classifier Collaborationcited by 97
  3. 2024Global and Local Prompts Cooperation via Optimal Transport for Federated Learningcited by 45
  4. 2022Test-Time Robust Personalization for Federated Learningcited by 39
  5. 2022FedTP: Federated Learning by Transformer Personalizationcited by 36
  6. 2022Motley: Benchmarking Heterogeneity and Personalization in Federated Learningcited by 31
  7. 2023Personalized Decentralized Federated Learning with Knowledge Distillationcited by 21
  8. 2023PeFLL: Personalized Federated Learning by Learning to Learncited by 21
  9. 2023Confidence-aware Personalized Federated Learning via Variational Expectation Maximizationcited by 17
  10. 2023Federated Learning for Data Streamscited by 15
  11. 2023PerAda: Parameter-Efficient Federated Learning Personalization with Generalization Guaranteescited by 15
  12. 2024MH-pFLID: Model Heterogeneous personalized Federated Learning via Injection and Distillation for Medical Data Analysiscited by 14
38 more
  1. 2023Provably Personalized and Robust Federated Learningcited by 11
  2. 2023FedDWA: Personalized Federated Learning with Dynamic Weight Adjustmentcited by 9
  3. 2023Profit: Benchmarking Personalization and Robustness Trade-off in Federated Prompt Tuningcited by 9
  4. 2022FedDAR: Federated Domain-Aware Representation Learningcited by 8
  5. 2024Communication-Efficient Personalized Federated Learning for Speech-to-Text Taskscited by 8
  6. 2024A Comprehensive Survey of Federated Transfer Learning: Challenges, Methods and Applicationscited by 8
  7. 2025dFLMoE: Decentralized Federated Learning via Mixture of Experts for Medical Data Analysiscited by 8
  8. 2023A Survey of Federated Evaluation in Federated Learningcited by 7
  9. 2023Fed-CO2: Cooperation of Online and Offline Models for Severe Data Heterogeneity in Federated Learningcited by 7
  10. 2024Balancing Similarity and Complementarity for Federated Learningcited by 7
  11. 2022Flow: Per-Instance Personalized Federated Learning Through Dynamic Routingcited by 6
  12. 2023Principled Federated Domain Adaptation: Gradient Projection and Auto-Weightingcited by 6
  13. 2023Federated Learning of Shareable Bases for Personalization-Friendly Image Classificationcited by 6
  14. 2024Gradient Coreset for Federated Learningcited by 6
  15. 2023A Practical Recipe for Federated Learning Under Statistical Heterogeneity Experimental Designcited by 5
  16. 2024pFLFE: Cross-silo Personalized Federated Learning via Feature Enhancement on Medical Image Segmentationcited by 5
  17. 2024Federated brain tumor segmentation: an extensive benchmarkcited by 5
  18. 2022Matching DNN Compression and Cooperative Training with Resources and Data Availabilitycited by 4
  19. 2024Incentivizing Truthful Collaboration in Heterogeneous Federated Learningcited by 3
  20. 2023Federated Variational Inference: Towards Improved Personalization and Generalizationcited by 2
  21. 2024Boosting the Performance of Decentralized Federated Learning via Catalyst Accelerationcited by 2
  22. 2024FedAli: Personalized Federated Learning Alignment with Prototype Layers for Generalized Mobile Servicescited by 2
  23. 2025Embedding-Based Federated Data Sharing via Differentially Private Conditional VAEscited by 2
  24. 2023PAGE: Equilibrate Personalization and Generalization in Federated Learningcited by 1
  25. 2023A Cautionary Tale: On the Role of Reference Data in Empirical Privacy Defensescited by 1
  26. 2024Learn What You Need in Personalized Federated Learningcited by 1
  27. 2024Context-Aware Orchestration of Energy-Efficient Gossip Learning Schemescited by 1
  28. 2024FedMeS: Personalized Federated Continual Learning Leveraging Local Memorycited by 1
  29. 2024FedCache 2.0: Federated Edge Learning with Knowledge Caching and Dataset Distillationcited by 1
  30. 2024Decentralized Personalized Federated Learningcited by 1
  31. 2025Federated Unlearning Made Practical: Seamless Integration via Negated Pseudo-Gradientscited by 1
  32. 2025FedOne: Query-Efficient Federated Learning for Black-box Discrete Prompt Learningcited by 1
  33. 2025Bayesian Coreset Optimization for Personalized Federated Learningcited by 1
  34. 2023Federated Deep Equilibrium Learning: Harnessing Compact Global Representations to Enhance Personalizationcited by 0
  35. 2023Factor-Assisted Federated Learning for Personalized Optimization with Heterogeneous Datacited by 0
  36. 2024Dependable Distributed Training of Compressed Machine Learning Modelscited by 0
  37. 2024Communication-Efficient and Privacy-Preserving Decentralized Meta-Learningcited by 0
  38. 2024Applied Federated Model Personalisation in the Industrial Domain: A Comparative Studycited by 0

50 most cited of 57 All 57 (JSON)