# Consensus-based Decentralized Multi-agent Reinforcement Learning for Random Access Network Optimization

arXiv: https://arxiv.org/abs/2508.07001 · HTML: https://mlsanity.com/paper/2508.07001/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 39 references that are in the map.
- First posted on arXiv 9 Aug 2025.

## Builds on (most cited first, 11 of 11)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents](https://mlsanity.com/paper/1802.08757/) (2018)
- [Finite-Time Error Bounds For Linear Stochastic Approximation and TD Learning](https://mlsanity.com/paper/1902.00923/) (2019)
- [Deep-Reinforcement Learning Multiple Access for Heterogeneous Wireless Networks](https://mlsanity.com/paper/1712.00162/) (2017)
- [Sample and Communication-Efficient Decentralized Actor-Critic Algorithms with Finite-Time Analysis](https://mlsanity.com/paper/2109.03699/) (2021)
- [Multi-Agent Deep Reinforcement Learning Multiple Access for Heterogeneous Wireless Networks with Imperfect Channels](https://mlsanity.com/paper/2003.11210/) (2020)
- [Sample and Communication Efficient Fully Decentralized MARL Policy Evaluation via a New Approach: Local TD update](https://mlsanity.com/paper/2403.15935/) (2024)
- [AI-Assisted MAC for Reconfigurable Intelligent Surface-Aided Wireless Networks: Challenges and Opportunities](https://mlsanity.com/paper/2105.00437/) (2021)
- [Enhancing WiFi Multiple Access Performance with Federated Deep Reinforcement Learning](https://mlsanity.com/paper/2102.07019/) (2021)
- [A survey on Machine Learning-based Performance Improvement of Wireless Networks: PHY, MAC and Network layer](https://mlsanity.com/paper/2001.04561/) (2020)

## Built on by (most cited first, 3 of 3)

- [Scalable Constrained Multi-Agent Reinforcement Learning via State Augmentation and Consensus for Separable Dynamics](https://mlsanity.com/paper/2605.30461/) (2026)
- [Learning When to Act: Interval-Aware Reinforcement Learning with Predictive Temporal Structure](https://mlsanity.com/paper/2603.22384/) (2026)
- [Distributed Direct Preference Optimization](https://mlsanity.com/paper/2605.20696/) (2026)

## Abstract (arXiv)

With wireless devices increasingly forming a unified smart network for seamless, user-friendly operations, random access (RA) medium access control (MAC) design is considered a key solution for handling unpredictable data traffic from multiple terminals. However, it remains challenging to design an effective RA-based MAC protocol to minimize collisions and ensure transmission fairness across the devices. While existing multi-agent reinforcement learning (MARL) approaches with centralized training and decentralized execution (CTDE) have been proposed to optimize RA performance, their reliance on centralized training and the significant overhead required for information collection can make real-world applications unrealistic. In this work, we adopt a fully decentralized MARL architecture, where policy learning does not rely on centralized tasks but leverages consensus-based information exchanges across devices. We design our MARL algorithm over an actor-critic (AC) network and propose exchanging only local rewards to minimize communication overhead. Furthermore, we provide a theoretical proof of global convergence for our approach. Numerical experiments show that our proposed MARL algorithm can significantly improve RA network performance compared to other baselines.