# Deep Reinforcement Learning for Simultaneous Sensing and Channel Access in Cognitive Networks

arXiv: https://arxiv.org/abs/2110.14541 · HTML: https://mlsanity.com/paper/2110.14541/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 62 references that are in the map.
- First posted on arXiv 24 Oct 2021.

## Builds on (most cited first, 15 of 15)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation](https://mlsanity.com/paper/1604.06057/) (2016)
- [Applications of Deep Reinforcement Learning in Communications and Networking: A Survey](https://mlsanity.com/paper/1810.07862/) (2018)
- [Federated Learning: A Signal Processing Perspective](https://mlsanity.com/paper/2103.17150/) (2021)
- [Deep Reinforcement Learning for Dynamic Multichannel Access in Wireless Networks](https://mlsanity.com/paper/1802.06958/) (2018)
- [PoPS: Policy Pruning and Shrinking for Deep Reinforcement Learning](https://mlsanity.com/paper/2001.05012/) (2020)
- [Learning in Restless Multi-Armed Bandits via Adaptive Arm Sequencing Rules](https://mlsanity.com/paper/1906.08120/) (2019)
- [Distributed Learning over Markovian Fading Channels for Stable Spectrum Access](https://mlsanity.com/paper/2101.11292/) (2021)
- [A Deep Actor-Critic Reinforcement Learning Framework for Dynamic Multichannel Access](https://mlsanity.com/paper/1908.08401/) (2019)
- [Deep Reinforcement Learning for Dynamic Spectrum Sensing and Aggregation in Multi-Channel Wireless Networks](https://mlsanity.com/paper/2007.13965/) (2020)
- [An order optimal policy for exploiting idle spectrum in cognitive radio networks](https://mlsanity.com/paper/1709.00237/) (2017)
- [Actor-Critic Deep Reinforcement Learning for Dynamic Multichannel Access](https://mlsanity.com/paper/1810.03695/) (2018)
- [Distributed Deep Reinforcement Learning for Collaborative Spectrum Sharing](https://mlsanity.com/paper/2104.02059/) (2021)

## Built on by (most cited first, 8 of 8)

- [Multi-Flow Transmission in Wireless Interference Networks: A Convergent Graph Learning Approach](https://mlsanity.com/paper/2303.15544/) (2023)
- [SINR-Aware Deep Reinforcement Learning for Distributed Dynamic Channel Allocation in Cognitive Interference Networks](https://mlsanity.com/paper/2402.17773/) (2024)
- [Collaborative Wideband Spectrum Sensing and Scheduling for Networked UAVs in UTM Systems](https://mlsanity.com/paper/2308.05036/) (2023)
- [Federated Learning-based Collaborative Wideband Spectrum Sensing and Scheduling for UAVs in UTM Systems](https://mlsanity.com/paper/2406.01727/) (2024)
- [DeepOFW: Deep Learning-Driven OFDM-Flexible Waveform Modulation for Peak-to-Average Power Ratio Reduction](https://mlsanity.com/paper/2603.23544/) (2026)
- [Fair Dynamic Spectrum Access via Fully Decentralized Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2503.24296/) (2025)
- [Control-Optimized Deep Reinforcement Learning for Artificially Intelligent Autonomous Systems](https://mlsanity.com/paper/2507.00268/) (2025)
- [Distributed Learning in Markovian Restless Bandits over Interference Graphs for Stable Spectrum Sharing](https://mlsanity.com/paper/2512.17161/) (2025)

## Abstract (arXiv)

We consider the problem of dynamic spectrum access (DSA) in cognitive wireless networks, where only partial observations are available to the users due to narrowband sensing and transmissions. The cognitive network consists of primary users (PUs) and a secondary user (SU), which operate in a time duplexing regime. The traffic pattern for each PU is assumed to be unknown to the SU and is modeled as a finite-memory Markov chain. Since observations are partial, then both channel sensing and access actions affect the throughput. The objective is to maximize the SU's long-term throughput. To achieve this goal, we develop a novel algorithm that learns both access and sensing policies via deep Q-learning, dubbed Double Deep Q-network for Sensing and Access (DDQSA). To the best of our knowledge, this is the first paper that solves both sensing and access policies for DSA via deep Q-learning. Second, we analyze the optimal policy theoretically to validate the performance of DDQSA. Although the general DSA problem is P-SPACE hard, we derive the optimal policy explicitly for a common model of a cyclic user dynamics. Our results show that DDQSA learns a policy that implements both sensing and channel access, and significantly outperforms existing approaches.