# Adaptive Contention Window Design using Deep Q-learning

arXiv: https://arxiv.org/abs/2011.09418 · HTML: https://mlsanity.com/paper/2011.09418/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Nov 2020.

## Built on by (most cited first, 13 of 13)

- [Link Scheduling using Graph Neural Networks](https://mlsanity.com/paper/2109.05536/) (2021)
- [Power Allocation for Wireless Federated Learning using Graph Neural Networks](https://mlsanity.com/paper/2111.07480/) (2021)
- [Graph-based Algorithm Unfolding for Energy-aware Power Allocation in Wireless Networks](https://mlsanity.com/paper/2201.11799/) (2022)
- [Learning to Transmit with Provable Guarantees in Wireless Federated Learning](https://mlsanity.com/paper/2304.09329/) (2023)
- [Enhancement of High-definition Map Update Service Through Coverage-aware and Reinforcement Learning](https://mlsanity.com/paper/2402.14582/) (2024)
- [Stability Analysis of Unfolded WMMSE for Power Allocation](https://mlsanity.com/paper/2110.07471/) (2021)
- [Learning-based Autonomous Channel Access in the Presence of Hidden Terminals](https://mlsanity.com/paper/2207.03605/) (2022)
- [ReinWiFi: Application-Layer QoS Optimization of WiFi Networks with Reinforcement Learning](https://mlsanity.com/paper/2405.03526/) (2024)
- [Multi-agent Assessment with QoS Enhancement for HD Map Updates in a Vehicular Network](https://mlsanity.com/paper/2407.21460/) (2024)
- [Coverage-aware and Reinforcement Learning Using Multi-agent Approach for HD Map QoS in a Realistic Environment](https://mlsanity.com/paper/2408.03329/) (2024)
- [Robust MIMO Detection using Hypernetworks with Learned Regularizers](https://mlsanity.com/paper/2110.07053/) (2021)
- [Sparsity-Aware Intelligent Massive Random Access Control in Open RAN: A Reinforcement Learning Based Approach](https://mlsanity.com/paper/2303.02657/) (2023)
- [AI-Enhanced Distributed Channel Access for Collision Avoidance in Future Wi-Fi 8](https://mlsanity.com/paper/2509.23154/) (2025)

## Abstract (arXiv)

We study the problem of adaptive contention window (CW) design for random-access wireless networks. More precisely, our goal is to design an intelligent node that can dynamically adapt its minimum CW (MCW) parameter to maximize a network-level utility knowing neither the MCWs of other nodes nor how these change over time. To achieve this goal, we adopt a reinforcement learning (RL) framework where we circumvent the lack of system knowledge with local channel observations and we reward actions that lead to high utilities. To efficiently learn these preferred actions, we follow a deep Q-learning approach, where the Q-value function is parametrized using a multi-layer perception. In particular, we implement a rainbow agent, which incorporates several empirical improvements over the basic deep Q-network. Numerical experiments based on the NS3 simulator reveal that the proposed RL agent performs close to optimal and markedly improves upon existing learning and non-learning based alternatives.