# Fully Distributed Online Training of Graph Neural Networks in Networked Systems

arXiv: https://arxiv.org/abs/2412.06105 · HTML: https://mlsanity.com/paper/2412.06105/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 25 references that are in the map.
- First posted on arXiv 8 Dec 2024.

## Builds on (most cited first, 17 of 17)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [A Survey on Distributed Machine Learning](https://mlsanity.com/paper/1912.09789/) (2019)
- [Convolutional Neural Network Architectures for Signals Supported on Graphs](https://mlsanity.com/paper/1805.00165/) (2018)
- [Multi-Agent Deep Reinforcement Learning for Dynamic Power Allocation in Wireless Networks](https://mlsanity.com/paper/1808.00490/) (2018)
- [Unfolding WMMSE using Graph Neural Networks for Efficient Power Allocation](https://mlsanity.com/paper/2009.10812/) (2020)
- [Graph Neural Networks for Wireless Communications: From Theory to Practice](https://mlsanity.com/paper/2203.10800/) (2022)
- [Distributed Graph Neural Network Training: A Survey](https://mlsanity.com/paper/2211.00216/) (2022)
- [DADAM: A Consensus-based Distributed Adaptive Gradient Method for Online Optimization](https://mlsanity.com/paper/1901.09109/) (2019)
- [Link Scheduling using Graph Neural Networks](https://mlsanity.com/paper/2109.05536/) (2021)
- [On the Convergence of Decentralized Adaptive Gradient Methods](https://mlsanity.com/paper/2109.03194/) (2021)
- [A Comprehensive Survey on Distributed Training of Graph Neural Networks](https://mlsanity.com/paper/2211.05368/) (2022)
- [Distributed Stochastic Gradient Descent: Nonconvexity, Nonsmoothness, and Convergence to Local Minima](https://mlsanity.com/paper/2003.02818/) (2020)
- [Congestion-aware Distributed Task Offloading in Wireless Multi-hop Networks Using Graph Neural Networks](https://mlsanity.com/paper/2312.02471/) (2023)
- [GLANCE: Graph-based Learnable Digital Twin for Communication Networks](https://mlsanity.com/paper/2408.09040/) (2024)
- [Biased Backpressure Routing Using Link Features and Graph Neural Networks](https://mlsanity.com/paper/2407.09753/) (2024)

## Built on by (most cited first, 3 of 3)

- [Graph Neural Network-Based Distributed Optimal Control for Linear Networked Systems: An Online Distributed Training Approach](https://mlsanity.com/paper/2504.06439/) (2025)
- [From Moments to Models: Graphon-Mixture Learning for Mixup and Contrastive Learning](https://mlsanity.com/paper/2510.03690/) (2025)
- [Graph Neural Network-Based Multicast Routing for On-Demand Streaming Services in 6G Networks](https://mlsanity.com/paper/2510.11109/) (2025)

## Abstract (arXiv)

Graph neural networks (GNNs) are powerful tools for developing scalable, decentralized artificial intelligence in large-scale networked systems, such as wireless networks, power grids, and transportation networks. Currently, GNNs in networked systems mostly follow a paradigm of `centralized training, distributed execution', which limits their adaptability and slows down their development cycles. In this work, we fill this gap for the first time by developing a communication-efficient, fully distributed online training approach for GNNs applied to large networked systems. For a mini-batch with B samples, our approach of training an L-layer GNN only adds L rounds of message passing to the LB rounds required by GNN inference, with doubled message sizes. Through numerical experiments in graph-based node regression, power allocation, and link scheduling in wireless networks, we demonstrate the effectiveness of our approach in training GNNs under supervised, unsupervised, and reinforcement learning paradigms.