mlsanityData to 7 Oct 2026

pFedAFM: Adaptive Feature Mixture for Batch-Level Personalization in Heterogeneous Federated Learning

arXiv 2404.17847 ↗First posted 27 Apr 2024cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Model-heterogeneous personalized federated learning (MHPFL) enables FL clients to train structurally different personalized models on non-independent and identically distributed (non-IID) local data. Existing MHPFL methods focus on achieving client-level personalization, but cannot address batch-level data heterogeneity. To bridge this important gap, we propose a model-heterogeneous personalized Federated learning approach with Adaptive Feature Mixture (pFedAFM) for supervised learning tasks. It consists of three novel designs: 1) A sharing global homogeneous small feature extractor is assigned alongside each client's local heterogeneous model (consisting of a heterogeneous feature extractor and a prediction header) to facilitate cross-client knowledge fusion. The two feature extractors share the local heterogeneous model's prediction header containing rich personalized prediction knowledge to retain personalized prediction capabilities. 2) An iterative training strategy is designed to alternately train the global homogeneous small feature extractor and the local heterogeneous large model for effective global-local knowledge exchange. 3) A trainable weight vector is designed to dynamically mix the features extracted by both feature extractors to adapt to batch-level data heterogeneity. Theoretical analysis proves that pFedAFM can converge over time. Extensive experiments on 2 benchmark datasets demonstrate that it significantly outperforms 7 state-of-the-art MHPFL methods, achieving up to 7.93% accuracy improvement while incurring low communication and computation costs.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 33 of its 58 references that are in the map. First posted on arXiv 27 Apr 2024.

Builds on

33 of 58 references.

References · 33

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2016An overview of gradient descent optimization algorithmscited by 1,021
  4. 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 994
  5. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  6. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  7. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
  8. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 380
  9. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 349
  10. 2021Federated Learning on Non-IID Data: A Surveycited by 345
  11. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 334
  12. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 322
21 more
  1. 2018Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Datacited by 242
  2. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  3. 2021FedKD: Communication Efficient Federated Learning via Knowledge Distillationcited by 185
  4. 2020Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training with Non-IID Private Datacited by 125
  5. 2021Parameterized Knowledge Transfer for Personalized Federated Learningcited by 108
  6. 2022FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extractioncited by 102
  7. 2019Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfercited by 91
  8. 2022Heterogeneous Ensemble Knowledge Transfer for Training Large Models in Federated Learningcited by 81
  9. 2020Federated Mutual Learningcited by 70
  10. 2021FedAUX: Leveraging Unlabeled Auxiliary Data in Federated Learningcited by 67
  11. 2023FedGH: Heterogeneous Federated Learning with Generalized Global Headercited by 56
  12. 2021FedGEMS: Federated Learning of Larger Server Models via Selective Knowledge Fusioncited by 49
  13. 2021Fed2: Feature-Aligned Federated Learningcited by 42
  14. 2019Wireless Federated Distillation for Distributed Edge Learning with Heterogeneous Datacited by 39
  15. 2021FedZKT: Zero-Shot Knowledge Transfer towards Resource-Constrained Federated Learning with Heterogeneous On-Device Modelscited by 31
  16. 2023Towards Personalized Federated Learning via Heterogeneous Model Reassemblycited by 31
  17. 2021FedMatch: Federated Learning Over Heterogeneous Question Answering Datacited by 23
  18. 2022FedClassAvg: Local Representation Learning for Personalized Federated Learning on Heterogeneous Neural Networkscited by 23
  19. 2022Enhancing Heterogeneous Federated Learning with Knowledge Extraction and Multi-Model Fusioncited by 10
  20. 2023FedSSA: Semantic Similarity-based Aggregation for Efficient Model-Heterogeneous Personalized Federated Learningcited by 10
  21. 2023pFedES: Model Heterogeneous Personalized Federated Learning with Feature Extractor Sharingcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 1.

3 papers.

  1. 2024Federated Model Heterogeneous Matryoshka Representation Learningcited by 21
  2. 2025FedSKD: Aggregation-free Model-heterogeneous Federated Learning via Multi-dimensional Similarity Knowledge Distillation for Medical Image Classificationcited by 1
  3. 2026CRIP: Channel Level Representation Injection for Personalized One-Shot Federated Learningcited by 0