mlsanityData to 7 Oct 2026

Architecture Agnostic Federated Learning for Neural Networks

arXiv 2202.07757 ↗First posted 15 Feb 2022cs.LGcited by 30

Not on the map. Why?

Abstract (arXiv)

With growing concerns regarding data privacy and rapid increase in data volume, Federated Learning(FL) has become an important learning paradigm. However, jointly learning a deep neural network model in a FL setting proves to be a non-trivial task because of the complexities associated with the neural networks, such as varied architectures across clients, permutation invariance of the neurons, and presence of non-linear transformations in each layer. This work introduces a novel Federated Heterogeneous Neural Networks (FedHeNN) framework that allows each client to build a personalised model without enforcing a common architecture across clients. This allows each client to optimize with respect to local data and compute constraints, while still benefiting from the learnings of other (potentially more powerful) clients. The key idea of FedHeNN is to use the instance-level representations obtained from peer clients to guide the simultaneous training on each client. The extensive experimental results demonstrate that the FedHeNN framework is capable of learning better performing models on clients in both the settings of homogeneous and heterogeneous architectures across clients.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). Builds on 31 of its 45 references that are in the map. First posted on arXiv 15 Feb 2022.

Builds on

31 of 45 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  5. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  6. 2018LEAF: A Benchmark for Federated Settingscited by 780
  7. 2021Model-Contrastive Federated Learningcited by 762
  8. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  9. 2017Federated Multi-Task Learningcited by 669
  10. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  11. 2020Federated Learning with Matched Averagingcited by 564
  12. 2021Federated Learning Based on Dynamic Regularizationcited by 555
19 more
  1. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  2. 2020An Efficient Framework for Clustered Federated Learningcited by 505
  3. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
  4. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
  5. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 349
  6. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 264
  7. 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
  8. 2019Model Fusion via Optimal Transportcited by 217
  9. 2019Adaptive Gradient-Based Meta-Learning Methodscited by 183
  10. 2020FedBE: Making Bayesian Model Ensemble Applicable to Federated Learningcited by 176
  11. 2020FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Datacited by 143
  12. 2020Salvaging Federated Learning by Local Adaptationcited by 142
  13. 2020FedSplit: An algorithmic framework for fast federated optimizationcited by 108
  14. 2021Towards Fair Federated Learning with Zero-Shot Data Augmentationcited by 64
  15. 2020Federated Learning via Synthetic Datacited by 53
  16. 2019Semi-Cyclic Stochastic Gradient Descentcited by 40
  17. 2021Improving Transformers with Probabilistic Attention Keyscited by 28
  18. 2021WAFFLE: Weighted Averaging for Personalized Federated Learningcited by 9
  19. 2015Representational Distance Learning for Deep Neural Networkscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 12; 2024: 5; 2025: 4; 2026: 3.

30 papers.

Sort:

By area:

  1. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
  2. 2023FedGH: Heterogeneous Federated Learning with Generalized Global Headercited by 56
  3. 2022Personalized Subgraph Federated Learningcited by 54
  4. 2023DFRD: Data-Free Robustness Distillation for Heterogeneous Federated Learningcited by 23
  5. 2024Federated Model Heterogeneous Matryoshka Representation Learningcited by 21
  6. 2022Federated Self-supervised Learning for Heterogeneous Clientscited by 16
  7. 2023FedSSA: Semantic Similarity-based Aggregation for Efficient Model-Heterogeneous Personalized Federated Learningcited by 10
  8. 2022Completely Heterogeneous Federated Learningcited by 9
  9. 2023pFedES: Model Heterogeneous Personalized Federated Learning with Feature Extractor Sharingcited by 9
  10. 2023Collaborative Learning via Prediction Consensuscited by 8
  11. 2024A Comprehensive Survey of Federated Transfer Learning: Challenges, Methods and Applicationscited by 8
  12. 2022Decentralized Learning with Multi-Headed Distillationcited by 5
18 more
  1. 2023Personalised Federated Learning On Heterogeneous Feature Spacescited by 5
  2. 2025Robust Asymmetric Heterogeneous Federated Learning with Corrupted Clientscited by 5
  3. 2025From Laboratory to Real World: A New Benchmark Towards Privacy-Preserved Visible-Infrared Person Re-Identificationcited by 5
  4. 2023UNIDEAL: Curriculum Knowledge Distillation Federated Learningcited by 4
  5. 2023Privacy Preserving Bayesian Federated Learning in Heterogeneous Settingscited by 2
  6. 2023Secure and Fast Asynchronous Vertical Federated Learning via Cascaded Hybrid Optimizationcited by 2
  7. 2024Federated Learning for Estimating Heterogeneous Treatment Effectscited by 2
  8. 2025Mosaic: Data-Free Knowledge Distillation via Mixture-of-Experts for Heterogeneous Distributed Environmentscited by 2
  9. 2026Toward Enhancing Representation Learning in Federated Multi-Task Settingscited by 2
  10. 2022Feature Correlation-guided Knowledge Transfer for Federated Self-supervised Learningcited by 1
  11. 2024A Kernel Perspective on Distillation-based Collaborative Learningcited by 1
  12. 2025FedRE: A Representation Entanglement Framework for Model-Heterogeneous Federated Learningcited by 1
  13. 2022Federated Learning of Neural ODE Models with Different Iteration Countscited by 0
  14. 2023Global Layers: Non-IID Tabular Federated Learningcited by 0
  15. 2023Horizontal Federated Computer Visioncited by 0
  16. 2024Trust-free Personalized Decentralized Learningcited by 0
  17. 2026Federated Ensemble Learning with Progressive Model Personalizationcited by 0
  18. 2026COSMOS: Model-Agnostic Personalized Federated Learning with Clustered Server Models and Pseudo-Label-Only Communicationcited by 0