# Transferable Graphical MARL for Real-Time Estimation in Dynamic Wireless Networks

arXiv: https://arxiv.org/abs/2404.03227 · HTML: https://mlsanity.com/paper/2404.03227/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 58 references that are in the map.
- First posted on arXiv 4 Apr 2024.

## Builds on (most cited first, 26 of 26)

- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1803.11485/) (2018)
- [Value-Decomposition Networks For Cooperative Multi-Agent Learning](https://mlsanity.com/paper/1706.05296/) (2017)
- [Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Tasks](https://mlsanity.com/paper/2006.07869/) (2020)
- [A Survey and Critique of Multiagent Deep Reinforcement Learning](https://mlsanity.com/paper/1810.05587/) (2018)
- [Stability Properties of Graph Neural Networks](https://mlsanity.com/paper/1905.04497/) (2019)
- [Convergence and Stability of Graph Convolutional Networks on Large Random Graphs](https://mlsanity.com/paper/2006.01868/) (2020)
- [Multi-Agent Deep Reinforcement Learning for Dynamic Power Allocation in Wireless Networks](https://mlsanity.com/paper/1808.00490/) (2018)
- [Single and Multi-Agent Deep Reinforcement Learning for AI-Enabled Wireless Networks: A Tutorial](https://mlsanity.com/paper/2011.03615/) (2020)
- [Gated Graph Recurrent Neural Networks](https://mlsanity.com/paper/2002.01038/) (2020)
- [Transferability of Graph Neural Networks: an Extended Graphon Approach](https://mlsanity.com/paper/2109.10096/) (2021)
- [Transferability Properties of Graph Neural Networks](https://mlsanity.com/paper/2112.04629/) (2021)
- [Resource Management in Wireless Networks via Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/2002.06215/) (2020)
- [Deep Multiagent Reinforcement Learning: Challenges and Directions](https://mlsanity.com/paper/2106.15691/) (2021)
- [Graphon Signal Processing](https://mlsanity.com/paper/2003.05030/) (2020)
- [Deep Reinforcement Learning for Event-Driven Multi-Agent Decision Processes](https://mlsanity.com/paper/1709.06656/) (2017)
- [Graphon Filters: Graph Signal Processing in the Limit](https://mlsanity.com/paper/2003.02099/) (2020)
- [Minimizing the Age of Information in Wireless Networks with Stochastic Arrivals](https://mlsanity.com/paper/1905.07020/) (2019)
- [Monitoring Correlated Sources: AoI-based Scheduling is Nearly Optimal](https://mlsanity.com/paper/2312.16813/) (2023)
- [Age of Information in Random Access Channels](https://mlsanity.com/paper/1912.01473/) (2019)
- [Real-time Sampling and Estimation on Random Access Channels: Age of Information and Beyond](https://mlsanity.com/paper/2007.03652/) (2020)
- [Learning Connectivity for Data Distribution in Robot Teams](https://mlsanity.com/paper/2103.05091/) (2021)
- [Information Freshness in Multi-Hop Wireless Networks](https://mlsanity.com/paper/2111.09217/) (2021)
- [Age of Information in Multihop Multicast Networks](https://mlsanity.com/paper/1812.10455/) (2018)
- [Minimizing Age of Information in Spatially Distributed Random Access Wireless Networks](https://mlsanity.com/paper/2212.03998/) (2022)

## Built on by (most cited first, 1 of 1)

- [Decentralized Learning Strategies for Estimation Error Minimization with Graph Neural Networks](https://mlsanity.com/paper/2601.12662/) (2026)

## Abstract (arXiv)

We study real-time sampling and estimation of autoregressive Markovian sources in decentralized and dynamic multi-hop networks that share similar structures. Nodes cache neighboring samples and communicate over wireless collision channels. The objective is to minimize the time-average estimation error and/or the age of information under decentralized policies, which we address by developing a unified graphical multi-agent reinforcement learning framework. A key feature of the framework is its transferability, enabled by the fact that the number of trainable parameters is independent of the number of agents, allowing a learned policy to be directly deployed on dynamic yet structurally similar graphs without re-training. Building on this design, we establish rigorous theoretical guarantees on the transferability of the resulting policies. Numerical experiments demonstrate that (i) our method outperforms state-of-the-art baselines on dynamic graphs; (ii) the trained policies transfer well to larger networks, with performance gains increasing with the number of nodes; and (iii) incorporating recurrence is crucial, enhancing resilience to non-stationarity in both independent learning and centralized training with decentralized execution.