mlsanityData to 7 Oct 2026

Personalized Federated Learning via Heterogeneous Modular Networks

arXiv 2210.14830 ↗First posted 26 Oct 2022cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

Personalized Federated Learning (PFL) which collaboratively trains a federated model while considering local clients under privacy constraints has attracted much attention. Despite its popularity, it has been observed that existing PFL approaches result in sub-optimal solutions when the joint distribution among local clients diverges. To address this issue, we present Federated Modular Network (FedMN), a novel PFL approach that adaptively selects sub-modules from a module pool to assemble heterogeneous neural architectures for different clients. FedMN adopts a light-weighted routing hypernetwork to model the joint distribution on each client and produce the personalized selection of the module blocks for each client. To reduce the communication burden in existing FL, we develop an efficient way to interact between the clients and the server. We conduct extensive experiments on the real-world test beds and the results show both the effectiveness and efficiency of the proposed FedMN over the baselines.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 36 of its 47 references that are in the map. First posted on arXiv 26 Oct 2022.

Builds on

36 of 47 references.

References · 36

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  4. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  5. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  6. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
  7. 2018Federated Learning with Non-IID Datacited by 1,099
  8. 2018LEAF: A Benchmark for Federated Settingscited by 780
  9. 2017Federated Multi-Task Learningcited by 669
  10. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  11. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
  12. 2020An Efficient Framework for Clustered Federated Learningcited by 505
24 more
  1. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
  2. 2021Towards Personalized Federated Learningcited by 462
  3. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
  4. 2019Federated Learning in Mobile Edge Networks: A Comprehensive Surveycited by 407
  5. 2020Adaptive Personalized Federated Learningcited by 353
  6. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 321
  7. 2020Personalized Federated Learning: A Meta-Learning Approachcited by 309
  8. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 264
  9. 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 189
  10. 2019Adaptive Gradient-Based Meta-Learning Methodscited by 183
  11. 2015Conditional Computation in Neural Networks for faster modelscited by 166
  12. 2018Mitigating Sybils in Federated Learning Poisoningcited by 158
  13. 2019Federated Evaluation of On-device Personalizationcited by 147
  14. 2017Routing Networks: Adaptive Selection of Non-linear Functions for Multi-Task Learningcited by 125
  15. 2016Decentralized Collaborative Learning of Personalized Models over Networkscited by 73
  16. 2019Variational Federated Multi-Task Learningcited by 69
  17. 2019Hierarchically Structured Meta-learningcited by 67
  18. 2018Modular Networks: Learning to Decompose Neural Computationcited by 62
  19. 2019An Investigation Into On-device Personalization of End-to-end Automatic Speech Recognition Modelscited by 25
  20. 2018Modular Generative Adversarial Networkscited by 16
  21. 2019Structure Learning for Neural Module Networkscited by 4
  22. 2019Recognizing Variables from their Data via Deep Embeddings of Distributionscited by 4
  23. 2021Compositional Models: Multi-Task Learning and Knowledge Transfer with Modular Networkscited by 2
  24. 2020Coded Caching over Multicast Routing Networkscited by 1

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 2; 2024: 2; 2025: 3.

8 papers.

  1. 2023Personalized Federated Learning under Mixture of Distributionscited by 37
  2. 2023Provably Personalized and Robust Federated Learningcited by 11
  3. 2022Decentralized Learning with Multi-Headed Distillationcited by 5
  4. 2024FedSI: Federated Subnetwork Inference for Efficient Uncertainty Quantificationcited by 4
  5. 2025FedADP: Unified Model Aggregation for Federated Learning with Heterogeneous Model Architecturescited by 2
  6. 2025Beyond Static Knowledge Messengers: Towards Adaptive, Fair, and Scalable Federated Learning for Medical AIcited by 2
  7. 2024Remembering Transformer for Continual Learningcited by 1
  8. 2025Communication-Efficient Module-Wise Federated Learning for Grasp Pose Detection in Cluttered Environmentscited by 0