mlsanityData to 7 Oct 2026

Adaptive Expert Models for Personalization in Federated Learning

arXiv 2206.07832 ↗First posted 15 Jun 2022cs.LGcited by 4

On the map: Mixture-of-experts → Federated mixture-of-experts

Abstract (arXiv)

Federated Learning (FL) is a promising framework for distributed learning when data is private and sensitive. However, the state-of-the-art solutions in this framework are not optimal when data is heterogeneous and non-Independent and Identically Distributed (non-IID). We propose a practical and robust approach to personalization in FL that adjusts to heterogeneous and non-IID data by balancing exploration and exploitation of several global models. To achieve our aim of personalization, we use a Mixture of Experts (MoE) that learns to group clients that are similar to each other, while using the global models more efficiently. We show that our approach achieves an accuracy up to 29.78 % and up to 4.38 % better compared to a local model in a pathological non-IID setting, even though we tune our approach in the IID setting.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Federated mixture-of-experts, one of 12 directions in Mixture-of-experts. Builds on 16 of its 24 references that are in the map. First posted on arXiv 15 Jun 2022.

Builds on

16 of 24 references.

References · 16

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,140
  3. 2019Towards Federated Learning at Scale: System Designcited by 826
  4. 2018LEAF: A Benchmark for Federated Settingscited by 780
  5. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  6. 2020An Efficient Framework for Clustered Federated Learningcited by 505
  7. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
  8. 2018Tune: A Research Platform for Distributed Model Selection and Trainingcited by 395
  9. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 334
  10. 2018Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Datacited by 242
  11. 2020Federated learning with hierarchical clustering of local updates to improve training on non-IID datacited by 222
  12. 2019Federated Evaluation of On-device Personalizationcited by 147
4 more
  1. 2021FedProc: Prototypical Contrastive Federated Learning on Non-IID datacited by 80
  2. 2021Federated Multi-armed Bandits with Personalizationcited by 62
  3. 2019Private Federated Learning with Domain Adaptationcited by 36
  4. 2020Towards Non-I.I.D. and Invisible Data with FedNAS: Federated Deep Learning via Neural Architecture Searchcited by 30

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 1.

4 papers.

  1. 2025A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applicationscited by 91
  2. 2024FedMoE-DA: Federated Mixture of Experts via Domain Aware Fine-grained Aggregationcited by 11
  3. 2024Personalized Federated Fine-Tuning for LLMs via Data-Driven Heterogeneous Model Architecturescited by 10
  4. 2026Aggregation Alignment for Federated Learning with Mixture-of-Experts under Data Heterogeneitycited by 4

Similar, not cited

Close in wording, no citation either way

  1. 2020Specialized federated learning using a mixture of experts
  2. 2026FedCoE: Bridging Generalization and Personalization via Federated Coordinated Dual-level MoEs
  3. 2025FLEx: Personalized Federated Learning for Mixture-of-Experts LLMs via Expert Grafting
  4. 2025FedMerge: Federated Personalization via Model Merging
  5. 2026Global Federated Learning Strategies for Building Efficient Personalized Models
  6. 2024pFedMoE: Data-Level Personalization with Mixture of Experts for Model-Heterogeneous Personalized Federated Learning
  7. 2020PFL-MoE: Personalized Federated Learning Based on Mixture of Experts
  8. 2024FedMoE: Personalized Federated Learning via Heterogeneous Mixture of Experts