# Deep Reinforcement Learning for Wireless Sensor Scheduling in Cyber-Physical Systems

arXiv: https://arxiv.org/abs/1809.05149 · HTML: https://mlsanity.com/paper/1809.05149/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Sep 2018.

## Built on by (most cited first, 23 of 23)

- [NeurWIN: Neural Whittle Index Network For Restless Bandits Via Deep RL](https://mlsanity.com/paper/2110.02128/) (2021)
- [Improved Soft Actor-Critic: Mixing Prioritized Off-Policy Samples with On-Policy Experience](https://mlsanity.com/paper/2109.11767/) (2021)
- [Deep Learning for Wireless Networked Systems: a joint Estimation-Control-Scheduling Approach](https://mlsanity.com/paper/2210.00673/) (2022)
- [Structure-Enhanced DRL for Optimal Transmission Scheduling](https://mlsanity.com/paper/2212.12704/) (2022)
- [Deep Reinforcement Learning for Wireless Scheduling in Distributed Networked Control](https://mlsanity.com/paper/2109.12562/) (2021)
- [Semantic-Aware Remote Estimation of Multiple Markov Sources Under Constraints](https://mlsanity.com/paper/2403.16855/) (2024)
- [A Learning Approach for Joint Design of Event-triggered Control and Power-Efficient Resource Allocation](https://mlsanity.com/paper/2205.07070/) (2022)
- [Deep Index Policy for Multi-Resource Restless Matching Bandit and Its Application in Multi-Channel Scheduling](https://mlsanity.com/paper/2408.07205/) (2024)
- [Distributed Deep Reinforcement Learning for Collaborative Spectrum Sharing](https://mlsanity.com/paper/2104.02059/) (2021)
- [Structure-Enhanced Deep Reinforcement Learning for Optimal Transmission Scheduling](https://mlsanity.com/paper/2211.10827/) (2022)
- [Semantic-aware Transmission Scheduling: a Monotonicity-driven Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2305.13706/) (2023)
- [GINO-Q: Learning an Asymptotically Optimal Index Policy for Restless Multi-armed Bandits](https://mlsanity.com/paper/2408.09882/) (2024)
- [Belief-Conditioned One-Step Diffusion: Real-Time Trajectory Planning with Just-Enough Sensing](https://mlsanity.com/paper/2508.12166/) (2025)
- [Adaptive Stochastic ADMM for Decentralized Reinforcement Learning in Edge Industrial IoT](https://mlsanity.com/paper/2107.00481/) (2021)
- [A Deep Learning Approach To Estimation Using Measurements Received Over a Network](https://mlsanity.com/paper/2201.08020/) (2022)
- [MORTAR: A Model-based Runtime Action Repair Framework for AI-enabled Cyber-Physical Systems](https://mlsanity.com/paper/2408.03892/) (2024)
- [Communication-Control Codesign for Large-Scale Wireless Networked Control Systems](https://mlsanity.com/paper/2410.11316/) (2024)
- [Wireless Resource Allocation with Collaborative Distributed and Centralized DRL under Control Channel Attacks](https://mlsanity.com/paper/2411.10702/) (2024)
- [Goal-oriented Transmission Scheduling: Structure-guided DRL with a Unified Dual On-policy and Off-policy Approach](https://mlsanity.com/paper/2501.11921/) (2025)
- [Nash Equilibrium Between Consumer Electronic Devices and DoS Attacker for Distributed IoT-enabled RSE Systems](https://mlsanity.com/paper/2504.09415/) (2025)
- [Actor-Critic Learning for Risk-Constrained Linear Quadratic Regulation](https://mlsanity.com/paper/2510.22267/) (2025)
- [Adaptive Scheduling: A Reinforcement Learning Whittle Index Approach for Wireless Sensor Networks](https://mlsanity.com/paper/2601.01179/) (2026)
- [Beyond Freshness and Semantics: A Coupon-Collector Framework for Effective Status Updates](https://mlsanity.com/paper/2603.26998/) (2026)

## Abstract (arXiv)

In many Cyber-Physical Systems, we encounter the problem of remote state estimation of geographically distributed and remote physical processes. This paper studies the scheduling of sensor transmissions to estimate the states of multiple remote, dynamic processes. Information from the different sensors have to be transmitted to a central gateway over a wireless network for monitoring purposes, where typically fewer wireless channels are available than there are processes to be monitored. For effective estimation at the gateway, the sensors need to be scheduled appropriately, i.e., at each time instant one needs to decide which sensors have network access and which ones do not. To address this scheduling problem, we formulate an associated Markov decision process (MDP). This MDP is then solved using a Deep Q-Network, a recent deep reinforcement learning algorithm that is at once scalable and model-free. We compare our scheduling algorithm to popular scheduling algorithms such as round-robin and reduced-waiting-time, among others. Our algorithm is shown to significantly outperform these algorithms for many example scenarios.