mlsanityData to 7 Oct 2026

Federated Learning with Heterogeneous Architectures using Graph HyperNetworks

arXiv 2201.08459 ↗First posted 20 Jan 2022cs.LG cs.AIcited by 17

Not on the map. Why?

Abstract (arXiv)

Standard Federated Learning (FL) techniques are limited to clients with identical network architectures. This restricts potential use-cases like cross-platform training or inter-organizational collaboration when both data privacy and architectural proprietary are required. We propose a new FL framework that accommodates heterogeneous client architecture by adopting a graph hypernetwork for parameter sharing. A property of the graph hyper network is that it can adapt to various computational graphs, thereby allowing meaningful parameter sharing across models. Unlike existing solutions, our framework does not limit the clients to share the same architecture type, makes no use of external data and does not require clients to disclose their model architecture. Compared with distillation-based and non-graph hypernetwork baselines, our method performs notably better on standard benchmarks. We additionally show encouraging generalization performance to unseen architectures.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 41 of its 53 references that are in the map. First posted on arXiv 20 Jan 2022.

Builds on

41 of 53 references.

References · 41

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  6. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  7. 2016Densely Connected Convolutional Networkscited by 6,691
  8. 2017Inductive Representation Learning on Large Graphscited by 6,284
  9. 2018How Powerful are Graph Neural Networks?cited by 4,355
  10. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  11. 2019Advances and Open Problems in Federated Learningcited by 3,021
  12. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
29 more
  1. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
  2. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  3. 2018Federated Learning with Non-IID Datacited by 1,099
  4. 2016HyperNetworkscited by 1,003
  5. 2017ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseasescited by 969
  6. 2018Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networkscited by 878
  7. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  8. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  9. 2015Gated Graph Sequence Neural Networkscited by 792
  10. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  11. 2018Local SGD Converges Fast and Communicates Littlecited by 595
  12. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
  13. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 334
  14. 2021Personalized Federated Learning using Hypernetworkscited by 210
  15. 2020Federated Learning of a Mixture of Global and Local Modelscited by 208
  16. 2020Personalized Federated Learning with First Order Model Optimizationcited by 201
  17. 2017SMASH: One-Shot Model Architecture Search through HyperNetworkscited by 200
  18. 2015Federated Optimization:Distributed Optimization Beyond the Datacentercited by 193
  19. 2019Privacy-preserving Federated Brain Tumour Segmentationcited by 169
  20. 2018cpSGD: Communication-efficient and differentially-private distributed SGDcited by 157
  21. 2020Survey of Personalization Techniques for Federated Learningcited by 144
  22. 2018Graph HyperNetworks for Neural Architecture Searchcited by 131
  23. 2020From Local Structures to Size Generalization in Graph Neural Networkscited by 119
  24. 2016Resnet in Resnet: Generalizing Residual Architecturescited by 110
  25. 2021Parameter Prediction for Unseen Deep Architecturescited by 89
  26. 2019Deep Meta Functionals for Shape Representationcited by 53
  27. 2019Federated Heavy Hitters Discovery with Differential Privacycited by 32
  28. 2019Hyper-Sphere Quantization: Communication-Efficient SGD for Federated Learningcited by 18
  29. 2020Molecule Property Prediction and Classification with Graph Hypernetworkscited by 5

Built on by

Cited by year

Citing papers per year: 2022: 5; 2023: 5; 2024: 3; 2025: 4.

17 papers.

Sort:

  1. 2023Equivariant Architectures for Learning in Deep Weight Spacescited by 106
  2. 2023A Brief Review of Hypernetworks in Deep Learningcited by 102
  3. 2023Graph Metanetworks for Processing Diverse Neural Architecturescited by 55
  4. 2022Federated Graph Neural Networks: Overview, Techniques and Challengescited by 52
  5. 2022Federated Learning Attacks and Defenses: A Surveycited by 11
  6. 2023Dynamic Inter-treatment Information Sharing for Individualized Treatment Effects Estimationcited by 11
  7. 2024Effective Heterogeneous Federated Learning via Efficient Hypernetwork-based Weight Generationcited by 9
  8. 2025A Comprehensive Data-centric Overview of Federated Graph Learningcited by 7
  9. 2024Sheaf HyperNetworks for Personalized Federated Learningcited by 5
  10. 2022FedorAS: Federated Architecture Search under system heterogeneitycited by 4
  11. 2024Edge Graph Intelligence: Reciprocally Empowering Edge Networks with Graph Intelligencecited by 2
  12. 2025FedAGHN: Personalized Federated Learning with Attentive Graph HyperNetworkscited by 1
5 more
  1. 2025Hypernetworks for Model-Heterogeneous Personalized Federated Learningcited by 1
  2. 2022Learning to reason about and to act on physical cascading eventscited by 0
  3. 2022M3FGM:a node masking and multi-granularity message passing-based federated graph model for spatial-temporal data predictioncited by 0
  4. 2023Replica Tree-based Federated Learning using Limited Datacited by 0
  5. 2025Moss: Proxy Model-based Full-Weight Aggregation in Federated Learning with Heterogeneous Modelscited by 0