Federated Graph Representation Learning using Self-Supervision
Not on the map. Why?
Abstract (arXiv)
Federated graph representation learning (FedGRL) brings the benefits of distributed training to graph structured data while simultaneously addressing some privacy and compliance concerns related to data curation. However, several interesting real-world graph data characteristics viz. label deficiency and downstream task heterogeneity are not taken into consideration in current FedGRL setups. In this paper, we consider a realistic and novel problem setting, wherein cross-silo clients have access to vast amounts of unlabeled data with limited or no labeled data and additionally have diverse downstream class label domains. We then propose a novel FedGRL formulation based on model interpolation where we aim to learn a shared global model that is optimized collaboratively using a self-supervised objective and gets downstream task supervision through local client models. We provide a specific instantiation of our general formulation using BGRL a SoTA self-supervised graph representation learning method and we empirically verify its effectiveness through realistic cross-slio datasets: (1) we adapt the Twitch Gamer Network which naturally simulates a cross-geo scenario and show that our formulation can provide consistent and avg. 6.1% gains over traditional supervised federated learning objectives and on avg. 1.7% gains compared to individual client specific self-supervised training and (2) we construct and introduce a new cross-silo dataset called Amazon Co-purchase Networks that have both the characteristics of the motivated problem setting. And, we witness on avg. 11.5% gains over traditional supervised federated learning and on avg. 1.9% gains over individually trained self-supervised models. Both experimental results point to the effectiveness of our proposed formulation. Finally, both our novel problem setting and dataset contributions provide new avenues for the research in FedGRL.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 48 of its 62 references that are in the map. First posted on arXiv 27 Oct 2022.
Builds on
48 of 62 references.
References · 48
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
- 2017Graph Attention Networkscited by 8,923
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
- 2017Inductive Representation Learning on Large Graphscited by 6,320
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
- 2018How Powerful are Graph Neural Networks?cited by 4,393
- 2019Advances and Open Problems in Federated Learningcited by 3,046
- 2018Federated Optimization in Heterogeneous Networkscited by 2,987
- 2020Exploring Simple Siamese Representation Learningcited by 2,866
- 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
36 more
- 2016Variational Graph Auto-Encoderscited by 1,382
- 2020Graph Contrastive Learning with Augmentationscited by 1,208
- 2018Deep Graph Infomaxcited by 1,106
- 2019Strategies for Pre-training Graph Neural Networkscited by 971
- 2018Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networkscited by 881
- 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
- 2020Contrastive Multi-View Representation Learning on Graphscited by 652
- 2019Multi-scale Attributed Node Embeddingcited by 548
- 2020Deep Graph Contrastive Representation Learningcited by 539
- 2020Graph Contrastive Learning with Adaptive Augmentationcited by 519
- 2020GCC: Graph Contrastive Coding for Graph Neural Network Pre-Trainingcited by 463
- 2019InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximizationcited by 441
- 2020Personalized Federated Learning: A Meta-Learning Approachcited by 312
- 2021Graph Self-Supervised Learning: A Surveycited by 289
- 2020Graph Representation Learning via Graphical Mutual Information Maximizationcited by 217
- 2021Adversarial Graph Augmentation to Improve Graph Contrastive Learningcited by 200
- 2021Self-Supervised Learning of Graph Neural Networks: A Unified Reviewcited by 184
- 2020Machine Learning on Graphs: A Model and Comprehensive Taxonomycited by 156
- 2021Subgraph Federated Learning with Missing Neighbor Generationcited by 132
- 2016Discriminative Embeddings of Latent Variable Models for Structured Datacited by 131
- 2021Federated Graph Classification over Non-IID Graphscited by 114
- 2020Federated Unsupervised Representation Learningcited by 93
- 2020Sub-graph Contrast for Scalable Self-Supervised Graph Representation Learningcited by 90
- 2022Divergence-aware Federated Self-Supervised Learningcited by 73
- 2021Collaborative Unsupervised Visual Representation Learning from Decentralized Datacited by 69
- 2020GraphFL: A Federated Learning Framework for Semi-Supervised Node Classification on Graphscited by 64
- 2020Vertically Federated Graph Neural Network for Privacy-Preserving Node Classificationcited by 61
- 2021Differentially Private Federated Knowledge Graphs Embeddingcited by 45
- 2022Orchestra: Unsupervised Federated Learning via Globally Consistent Clusteringcited by 45
- 2020ASFGNN: Automated Separated-Federated Graph Neural Networkcited by 36
- 2021FedGraph: Federated Graph Learning with Intelligent Samplingcited by 34
- 2021SSFL: Tackling Label Deficiency in Federated Learning via Personalized Self-Supervisioncited by 27
- 2021Self-supervised Graph Neural Networks without explicit negative samplingcited by 25
- 2021A Vertical Federated Learning Framework for Graph Convolutional Networkcited by 21
- 2018Prioritizing network communitiescited by 14
- 2020Self-supervised Graph Representation Learning via Bootstrappingcited by 8
Built on by
Not cited yet.