# A Deep Reinforcement Learning Framework for Contention-Based Spectrum Sharing

arXiv: https://arxiv.org/abs/2110.02736 · HTML: https://mlsanity.com/paper/2110.02736/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 41 references that are in the map.
- First posted on arXiv 5 Oct 2021.

## Builds on (most cited first, 11 of 11)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Dueling Network Architectures for Deep Reinforcement Learning](https://mlsanity.com/paper/1511.06581/) (2015)
- [Learning to Communicate with Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1605.06676/) (2016)
- [Deep Recurrent Q-Learning for Partially Observable MDPs](https://mlsanity.com/paper/1507.06527/) (2015)
- [The Complexity of Decentralized Control of Markov Decision Processes](https://mlsanity.com/paper/1301.3836/) (2013)
- [Multiagent Cooperation and Competition with Deep Reinforcement Learning](https://mlsanity.com/paper/1511.08779/) (2015)
- [Resource Management in Wireless Networks via Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/2002.06215/) (2020)
- [5G New Radio: Unveiling the Essentials of the Next Generation Wireless Access Technology](https://mlsanity.com/paper/1806.06898/) (2018)
- [Distributive Dynamic Spectrum Access through Deep Reinforcement Learning: A Reservoir Computing Based Approach](https://mlsanity.com/paper/1810.11758/) (2018)
- [Multi-Agent Deep Reinforcement Learning based Spectrum Allocation for D2D Underlay Communications](https://mlsanity.com/paper/1912.09302/) (2019)
- [Next Generation Wi-Fi and 5G NR-U in the 6 GHz Bands: Opportunities & Challenges](https://mlsanity.com/paper/2006.16534/) (2020)

## Built on by (most cited first, 3 of 3)

- [Learning Resilient Radio Resource Management Policies with Graph Neural Networks](https://mlsanity.com/paper/2203.11012/) (2022)
- [Self-Sustaining Multiple Access with Continual Deep Reinforcement Learning for Dynamic Metaverse Applications](https://mlsanity.com/paper/2309.10177/) (2023)
- [Distributed Proximal Policy Optimization for Contention-Based Spectrum Access](https://mlsanity.com/paper/2111.09420/) (2021)

## Abstract (arXiv)

The increasing number of wireless devices operating in unlicensed spectrum motivates the development of intelligent adaptive approaches to spectrum access. We consider decentralized contention-based medium access for base stations (BSs) operating on unlicensed shared spectrum, where each BS autonomously decides whether or not to transmit on a given resource. The contention decision attempts to maximize not its own downlink throughput, but rather a network-wide objective. We formulate this problem as a decentralized partially observable Markov decision process with a novel reward structure that provides long term proportional fairness in terms of throughput. We then introduce a two-stage Markov decision process in each time slot that uses information from spectrum sensing and reception quality to make a medium access decision. Finally, we incorporate these features into a distributed reinforcement learning framework for contention-based spectrum access. Our formulation provides decentralized inference, online adaptability and also caters to partial observability of the environment through recurrent Q-learning. Empirically, we find its maximization of the proportional fairness metric to be competitive with a genie-aided adaptive energy detection threshold, while being robust to channel fading and small contention windows.