# Learning-based Autonomous Channel Access in the Presence of Hidden Terminals

arXiv: https://arxiv.org/abs/2207.03605 · HTML: https://mlsanity.com/paper/2207.03605/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 42 references that are in the map.
- First posted on arXiv 7 Jul 2022.

## Builds on (most cited first, 16 of 16)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Counterfactual Multi-Agent Policy Gradients](https://mlsanity.com/paper/1705.08926/) (2017)
- [QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1803.11485/) (2018)
- [Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms](https://mlsanity.com/paper/1911.10635/) (2019)
- [Deep Recurrent Q-Learning for Partially Observable MDPs](https://mlsanity.com/paper/1507.06527/) (2015)
- [Deep-Reinforcement Learning Multiple Access for Heterogeneous Wireless Networks](https://mlsanity.com/paper/1712.00162/) (2017)
- [Towards Joint Learning of Optimal MAC Signaling and Wireless Channel Access](https://mlsanity.com/paper/2007.09948/) (2020)
- [The Emergence of Wireless MAC Protocols with Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2108.07144/) (2021)
- [Adaptive Contention Window Design using Deep Q-learning](https://mlsanity.com/paper/2011.09418/) (2020)
- [A Deep Actor-Critic Reinforcement Learning Framework for Dynamic Multichannel Access](https://mlsanity.com/paper/1908.08401/) (2019)
- [Contention Window Optimization in IEEE 802.11ax Networks with Deep Reinforcement Learning](https://mlsanity.com/paper/2003.01492/) (2020)
- [Multi-Agent Deep Reinforcement Learning Multiple Access for Heterogeneous Wireless Networks with Imperfect Channels](https://mlsanity.com/paper/2003.11210/) (2020)
- [Partially Observable Minimum-Age Scheduling: The Greedy Policy](https://mlsanity.com/paper/2009.13441/) (2020)
- [Non-Uniform Time-Step Deep Q-Network for Carrier-Sense Multiple Access in Heterogeneous Wireless Networks](https://mlsanity.com/paper/1910.05221/) (2019)
- [Sporadic Ultra-Time-Critical Crowd Messaging in V2X](https://mlsanity.com/paper/2003.01906/) (2020)

## Built on by (most cited first, 1 of 1)

- [Decentralized Channel Management in WLANs with Graph Neural Networks](https://mlsanity.com/paper/2210.16949/) (2022)

## Abstract (arXiv)

We consider the problem of autonomous channel access (AutoCA), where a group of terminals tries to discover a communication strategy with an access point (AP) via a common wireless channel in a distributed fashion. Due to the irregular topology and the limited communication range of terminals, a practical challenge for AutoCA is the hidden terminal problem, which is notorious in wireless networks for deteriorating the throughput and delay performances. To meet the challenge, this paper presents a new multi-agent deep reinforcement learning paradigm, dubbed MADRL-HT, tailored for AutoCA in the presence of hidden terminals. MADRL-HT exploits topological insights and transforms the observation space of each terminal into a scalable form independent of the number of terminals. To compensate for the partial observability, we put forth a look-back mechanism such that the terminals can infer behaviors of their hidden terminals from the carrier sensed channel states as well as feedback from the AP. A window-based global reward function is proposed, whereby the terminals are instructed to maximize the system throughput while balancing the terminals' transmission opportunities over the course of learning. Extensive numerical experiments verified the superior performance of our solution benchmarked against the legacy carrier-sense multiple access with collision avoidance (CSMA/CA) protocol.