mlsanityData to 14 Oct 2026

Sheaf HyperNetworks for Personalized Federated Learning

arXiv 2405.20882 ↗First posted 31 May 2024cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Graph hypernetworks (GHNs), constructed by combining graph neural networks (GNNs) with hypernetworks (HNs), leverage relational data across various domains such as neural architecture search, molecular property prediction and federated learning. Despite GNNs and HNs being individually successful, we show that GHNs present problems compromising their performance, such as over-smoothing and heterophily. Moreover, we cannot apply GHNs directly to personalized federated learning (PFL) scenarios, where a priori client relation graph may be absent, private, or inaccessible. To mitigate these limitations in the context of PFL, we propose a novel class of HNs, sheaf hypernetworks (SHNs), which combine cellular sheaf theory with HNs to improve parameter sharing for PFL. We thoroughly evaluate SHNs across diverse PFL tasks, including multi-class classification, traffic and weather forecasting. Additionally, we provide a methodology for constructing client relation graphs in scenarios where such graphs are unavailable. We show that SHNs consistently outperform existing PFL solutions in complex non-IID scenarios. While the baselines' performance fluctuates depending on the task, SHNs show improvements of up to 2.7% in accuracy and 5.3% in lower mean squared error over the best-performing baseline.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 36 of its 52 references that are in the map. First posted on arXiv 31 May 2024.

Builds on

36 of 52 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  4. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  5. 2019Advances and Open Problems in Federated Learningcited by 3,046
  6. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  7. 2016HyperNetworkscited by 1,027
  8. 2020Simple and Deep Graph Convolutional Networkscited by 860
  9. 2020The Future of Digital Health with Federated Learningcited by 633
  10. 2020Personalized Federated Learning with Moreau Envelopescited by 618
  11. 2018Federated Learning for Mobile Keyboard Predictioncited by 572
  12. 2021Towards Personalized Federated Learningcited by 464
24 more
  1. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 387
  2. 2019A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protectioncited by 374
  3. 2020Personalized Federated Learning: A Meta-Learning Approachcited by 312
  4. 2019PairNorm: Tackling Oversmoothing in GNNscited by 297
  5. 2020A Note on Over-Smoothing for Graph Neural Networkscited by 249
  6. 2023A Survey on Oversmoothing in Graph Neural Networkscited by 247
  7. 2022Neural Sheaf Diffusion: A Topological Perspective on Heterophily and Oversmoothing in GNNscited by 214
  8. 2021Personalized Federated Learning using Hypernetworkscited by 212
  9. 2017SMASH: One-Shot Model Architecture Search through HyperNetworkscited by 202
  10. 2015Federated Optimization:Distributed Optimization Beyond the Datacentercited by 195
  11. 2018Graph HyperNetworks for Neural Architecture Searchcited by 132
  12. 2015A Large-Scale Car Dataset for Fine-Grained Categorization and Verificationcited by 130
  13. 2022Pre-training Enhanced Spatial-temporal Graph Neural Network for Multivariate Time Series Forecastingcited by 117
  14. 2023A Brief Review of Hypernetworks in Deep Learningcited by 102
  15. 2020Sheaf Neural Networkscited by 84
  16. 2021Cross-Node Federated Graph Neural Network for Spatio-Temporal Data Modelingcited by 69
  17. 2022Sheaf Neural Networks with Connection Laplacianscited by 65
  18. 2022ZeroFL: Efficient On-Device Training for Federated Learning with Local Sparsitycited by 45
  19. 2023Sheaf Hypergraph Networkscited by 34
  20. 2022Personalized Federated Learning With Graphcited by 29
  21. 2022Federated Learning with Heterogeneous Architectures using Graph HyperNetworkscited by 19
  22. 2022Graph-Relational Domain Adaptationcited by 19
  23. 2023Efficient and Secure Federated Learning for Financial Applicationscited by 17
  24. 2020Molecule Property Prediction and Classification with Graph Hypernetworkscited by 5

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 3.

5 papers.

  1. 2024The Heterophilic Graph Learning Handbook: Benchmarks, Models, Theoretical Analysis, Applications and Challengescited by 29
  2. 2025Copresheaf Topological Neural Networks: A Generalized Deep Learning Frameworkcited by 12
  3. 2024Heterogeneous Sheaf Neural Networkscited by 10
  4. 2025Polynomial Neural Sheaf Diffusion: A Spectral Filtering Approach on Cellular Sheavescited by 9
  5. 2025On the Sheafification of Higher-Order Message Passingcited by 1