# Cooperative Multi-Agent Reinforcement Learning Based Distributed Dynamic Spectrum Access in Cognitive Radio Networks

arXiv: https://arxiv.org/abs/2106.09274 · HTML: https://mlsanity.com/paper/2106.09274/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 36 references that are in the map.
- First posted on arXiv 17 Jun 2021.

## Builds on (most cited first, 12 of 12)

- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1803.11485/) (2018)
- [Deep Recurrent Q-Learning for Partially Observable MDPs](https://mlsanity.com/paper/1507.06527/) (2015)
- [Applications of Deep Reinforcement Learning in Communications and Networking: A Survey](https://mlsanity.com/paper/1810.07862/) (2018)
- [Deep-Reinforcement Learning Multiple Access for Heterogeneous Wireless Networks](https://mlsanity.com/paper/1712.00162/) (2017)
- [Deep Reinforcement Learning for Dynamic Multichannel Access in Wireless Networks](https://mlsanity.com/paper/1802.06958/) (2018)
- [Artificial Intelligence-Enabled Intelligent 6G Networks](https://mlsanity.com/paper/1912.05744/) (2019)
- [Deep Reinforcement Learning Based Intelligent Reflecting Surface for Secure Wireless Communications](https://mlsanity.com/paper/2002.12271/) (2020)
- [Distributive Dynamic Spectrum Access through Deep Reinforcement Learning: A Reservoir Computing Based Approach](https://mlsanity.com/paper/1810.11758/) (2018)
- [A Deep Actor-Critic Reinforcement Learning Framework for Dynamic Multichannel Access](https://mlsanity.com/paper/1908.08401/) (2019)
- [Intelligent Reflecting Surface Assisted Anti-Jamming Communications: A Fast Reinforcement Learning Approach](https://mlsanity.com/paper/2004.12539/) (2020)

## Built on by (most cited first, 7 of 7)

- [DeepSeek-Inspired Exploration of RL-based LLMs and Synergy with Wireless Networks: A Survey](https://mlsanity.com/paper/2503.09956/) (2025)
- [Quantum Multi-Agent Actor-Critic Networks for Cooperative Mobile Access in Multi-UAV Systems](https://mlsanity.com/paper/2302.04445/) (2023)
- [A Multi-Agent Reinforcement Learning Testbed for Cognitive Radio Applications](https://mlsanity.com/paper/2410.21521/) (2024)
- [Deep Reinforcement Learning for Distributed and Uncoordinated Cognitive Radios Resource Allocation](https://mlsanity.com/paper/2205.13944/) (2022)
- [Two Tales of Platoon Intelligence for Autonomous Mobility Control: Enabling Deep Learning Recipes](https://mlsanity.com/paper/2307.09711/) (2023)
- [Deep Reinforcement Learning for Uplink Scheduling in NOMA-URLLC Networks](https://mlsanity.com/paper/2308.14523/) (2023)
- [Channel-Token Attention for Reliable Dynamic Spectrum Access under Bursty Primary-User Traffic](https://mlsanity.com/paper/2608.22992/) (2026)

## Abstract (arXiv)

With the development of the 5G and Internet of Things, amounts of wireless devices need to share the limited spectrum resources. Dynamic spectrum access (DSA) is a promising paradigm to remedy the problem of inefficient spectrum utilization brought upon by the historical command-and-control approach to spectrum allocation. In this paper, we investigate the distributed DSA problem for multi-user in a typical multi-channel cognitive radio network. The problem is formulated as a decentralized partially observable Markov decision process (Dec-POMDP), and we proposed a centralized off-line training and distributed on-line execution framework based on cooperative multi-agent reinforcement learning (MARL). We employ the deep recurrent Q-network (DRQN) to address the partial observability of the state for each cognitive user. The ultimate goal is to learn a cooperative strategy which maximizes the sum throughput of cognitive radio network in distributed fashion without coordination information exchange between cognitive users. Finally, we validate the proposed algorithm in various settings through extensive experiments. From the simulation results, we can observe that the proposed algorithm can converge fast and achieve almost the optimal performance.