mlsanityData to 7 Oct 2026

Completely Heterogeneous Federated Learning

arXiv 2210.15865 ↗First posted 28 Oct 2022cs.LG cs.CR cs.DCcited by 9

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) faces three major difficulties: cross-domain, heterogeneous models, and non-i.i.d. labels scenarios. Existing FL methods fail to handle the above three constraints at the same time, and the level of privacy protection needs to be lowered (e.g., the model architecture and data category distribution can be shared). In this work, we propose the challenging "completely heterogeneous" scenario in FL, which refers to that each client will not expose any private information including feature space, model architecture, and label distribution. We then devise an FL framework based on parameter decoupling and data-free knowledge distillation to solve the problem. Experiments show that our proposed method achieves high performance in completely heterogeneous scenarios where other approaches fail.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 22 of its 30 references that are in the map. First posted on arXiv 28 Oct 2022.

Builds on

22 of 30 references.

References · 22

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  5. 2019Advances and Open Problems in Federated Learningcited by 3,021
  6. 2019Searching for MobileNetV3cited by 1,984
  7. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,906
  8. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  9. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  10. 2019Federated Learning with Personalization Layerscited by 533
  11. 2021Towards Personalized Federated Learningcited by 462
  12. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
10 more
  1. 2022Decoupled Knowledge Distillationcited by 380
  2. 2015Federated Optimization:Distributed Optimization Beyond the Datacentercited by 193
  3. 2022Federated Learning with Partial Model Personalizationcited by 114
  4. 2021SubTab: Subsetting Features of Tabular Data for Self-Supervised Representation Learningcited by 101
  5. 2022Heterogeneous Ensemble Knowledge Transfer for Training Large Models in Federated Learningcited by 81
  6. 2021FedLab: A Flexible Federated Learning Frameworkcited by 54
  7. 2021FedZKT: Zero-Shot Knowledge Transfer towards Resource-Constrained Federated Learning with Heterogeneous On-Device Modelscited by 31
  8. 2022Architecture Agnostic Federated Learning for Neural Networkscited by 30
  9. 2021FedH2L: Federated Learning with Model and Statistical Heterogeneitycited by 17
  10. 2021Aggregate or Not? Exploring Where to Privatize in DNN Based Federated Learning Under Different Non-IID Scenescited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 4; 2025: 0; 2026: 1.

9 papers.

  1. 2023FedGH: Heterogeneous Federated Learning with Generalized Global Headercited by 56
  2. 2024Federated Model Heterogeneous Matryoshka Representation Learningcited by 21
  3. 2024FedMKT: Federated Mutual Knowledge Transfer for Large and Small Language Modelscited by 19
  4. 2023FedSSA: Semantic Similarity-based Aggregation for Efficient Model-Heterogeneous Personalized Federated Learningcited by 10
  5. 2023pFedES: Model Heterogeneous Personalized Federated Learning with Feature Extractor Sharingcited by 9
  6. 2024A Comprehensive Survey of Federated Transfer Learning: Challenges, Methods and Applicationscited by 8
  7. 2023Global Layers: Non-IID Tabular Federated Learningcited by 0
  8. 2024MAPL: Model Agnostic Peer-to-peer Learningcited by 0
  9. 2026Social Hippocampus Memory Learningcited by 0