mlsanityData to 7 Oct 2026

Think Locally, Act Globally: Federated Learning with Local and Global Representations

arXiv 2001.01523 ↗First posted 6 Jan 2020cs.LG cs.DC stat.MLcited by 322

Not on the map. Why?

Abstract (arXiv)

Federated learning is a method of training models on private data distributed over multiple devices. To keep device data private, the global model is trained by only communicating parameters and updates which poses scalability challenges for large models. To this end, we propose a new federated learning algorithm that jointly learns compact local representations on each device and a global model across all devices. As a result, the global model can be smaller since it only operates on local representations, reducing the number of communicated parameters. Theoretically, we provide a generalization analysis which shows that a combination of local and global models reduces both variance in the data as well as variance across device distributions. Empirically, we demonstrate that local models enable communication-efficient training while retaining performance. We also evaluate on the task of personalized mood prediction from real-world mobile data where privacy is key. Finally, local models handle heterogeneous data from new devices, and learn fair representations that obfuscate protected attributes such as race, age, and gender.

Key facts. Cited by 322 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Jan 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 55; 2022: 56; 2023: 68; 2024: 73; 2025: 46; 2026: 24.

322 papers.

Sort:

By area:

  1. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  2. 2021Towards Personalized Federated Learningcited by 462
  3. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 349
  4. 2021Federated Learning on Non-IID Data: A Surveycited by 345
  5. 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
  6. 2021Personalized Federated Learning using Hypernetworkscited by 210
  7. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  8. 2021FedBABU: Towards Enhanced Representation for Federated Image Classificationcited by 180
  9. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
  10. 2021MultiBench: Multiscale Benchmarks for Multimodal Representation Learningcited by 143
  11. 2021Federated Learning for Open Bankingcited by 138
  12. 2022PromptFL: Let Federated Participants Cooperatively Learn Prompts Instead of Models -- Federated Learning in Age of Foundation Modelcited by 133
38 more
  1. 2022Federated Learning with Partial Model Personalizationcited by 114
  2. 2021Multi-Center Federated Learning: Clients Clustering for Better Personalizationcited by 112
  3. 2021Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learningcited by 111
  4. 2023Personalized Federated Learning with Feature Alignment and Classifier Collaborationcited by 97
  5. 2023Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmarkcited by 97
  6. 2021Federated Reconstruction: Partially Local Federated Learningcited by 78
  7. 2021Personalized Federated Learning with Gaussian Processescited by 76
  8. 2022DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Trainingcited by 74
  9. 2021Cross-Node Federated Graph Neural Network for Spatio-Temporal Data Modelingcited by 69
  10. 2021What Do We Mean by Generalization in Federated Learning?cited by 62
  11. 2021Personalized Federated Learning through Local Memorizationcited by 57
  12. 2023FedGH: Heterogeneous Federated Learning with Generalized Global Headercited by 56
  13. 2022FedAvg with Fine Tuning: Local Updates Lead to Representation Learningcited by 55
  14. 2021Federated Dynamic Sparse Training: Computing Less, Communicating Less, Yet Learning Bettercited by 54
  15. 2022Personalized Subgraph Federated Learningcited by 54
  16. 2024FedTGP: Trainable Global Prototypes with Adaptive-Margin-Enhanced Contrastive Learning for Data and Model Heterogeneity in Federated Learningcited by 47
  17. 2021SemiFL: Semi-Supervised Federated Learning for Unlabeled Clients with Alternate Trainingcited by 45
  18. 2024Global and Local Prompts Cooperation via Optimal Transport for Federated Learningcited by 45
  19. 2022pFL-Bench: A Comprehensive Benchmark for Personalized Federated Learningcited by 44
  20. 2022A Survey on Heterogeneous Federated Learningcited by 42
  21. 2022Label-Efficient Self-Supervised Federated Learning for Tackling Data Heterogeneity in Medical Imagingcited by 39
  22. 2023Heterogeneous Federated Knowledge Graph Embedding Learning and Unlearningcited by 39
  23. 2023Generalizable Heterogeneous Federated Cross-Correlation and Instance Similarity Learningcited by 39
  24. 2023Efficient Personalized Federated Learning via Sparse Model-Adaptationcited by 38
  25. 2023pFedLoRA: Model-Heterogeneous Personalized Federated Learning with LoRA Tuningcited by 38
  26. 2022On Privacy and Personalization in Cross-Silo Federated Learningcited by 36
  27. 2022Efficient Distribution Similarity Identification in Clustered Federated Learning via Principal Angles Between Client Data Subspacescited by 36
  28. 2022FedTP: Federated Learning by Transformer Personalizationcited by 36
  29. 2021Emerging Trends in Federated Learning: From Model Fusion to Federated X Learningcited by 35
  30. 2021Specificity-Preserving Federated Learning for MR Image Reconstructioncited by 35
  31. 2022FedPop: A Bayesian Approach for Personalised Federated Learningcited by 35
  32. 2021Personalized Federated Learning: A Unified Framework and Universal Optimization Techniquescited by 34
  33. 2022CD2-pFed: Cyclic Distillation-guided Channel Decoupling for Model Personalization in Federated Learningcited by 34
  34. 2022Federated Learning of Generative Image Priors for MRI Reconstructioncited by 33
  35. 2021Multimodal Federated Learning on IoT Datacited by 32
  36. 2021FedHM: Efficient Federated Learning for Heterogeneous Models via Low-rank Factorizationcited by 31
  37. 2022On the Convergence of Clustered Federated Learningcited by 31
  38. 2022Motley: Benchmarking Heterogeneity and Personalization in Federated Learningcited by 31

50 most cited of 322 All 322 (JSON)