# Deep Reinforcement Learning for Wireless Scheduling in Distributed Networked Control

arXiv: https://arxiv.org/abs/2109.12562 · HTML: https://mlsanity.com/paper/2109.12562/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 72 references that are in the map.
- First posted on arXiv 26 Sep 2021.

## Builds on (most cited first, 19 of 19)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Deep Reinforcement Learning for Wireless Sensor Scheduling in Cyber-Physical Systems](https://mlsanity.com/paper/1809.05149/) (2018)
- [Deep Reinforcement Learning for Event-Triggered Control](https://mlsanity.com/paper/1809.05152/) (2018)
- [Wireless Network Design for Control Systems: A Survey](https://mlsanity.com/paper/1708.07353/) (2017)
- [DeepCAS: A Deep Reinforcement Learning Algorithm for Control-Aware Scheduling](https://mlsanity.com/paper/1803.02998/) (2018)
- [Remote State Estimation with Smart Sensors over Markov Fading Channels](https://mlsanity.com/paper/2005.07871/) (2020)
- [Control Aware Radio Resource Allocation in Low Latency Wireless Control Systems](https://mlsanity.com/paper/1811.00409/) (2018)
- [Model-Free Design of Control Systems over Wireless Fading Channels](https://mlsanity.com/paper/2009.01751/) (2020)
- [Optimal Sensor Scheduling for Multiple Linear Dynamical Systems](https://mlsanity.com/paper/1604.04158/) (2016)
- [Optimal Downlink-Uplink Scheduling of Wireless Networked Control for Industrial IoT](https://mlsanity.com/paper/1907.07841/) (2019)
- [On the Optimal Solutions of the Infinite-Horizon Linear Sensor Scheduling Problem](https://mlsanity.com/paper/1312.0157/) (2013)
- [Sensor Scheduling in Variance Based Event Triggered Estimation with Packet Drops](https://mlsanity.com/paper/1511.04792/) (2015)
- [Deep reinforcement learning for scheduling in large-scale networked control systems](https://mlsanity.com/paper/1905.05992/) (2019)
- [Infinite Horizon Optimal Transmission Power Control for Remote State Estimation over Fading Channels](https://mlsanity.com/paper/1604.08680/) (2016)
- [Real-Time Remote Estimation with Hybrid ARQ in Wireless Networked Control](https://mlsanity.com/paper/1903.12472/) (2019)
- [Stochastic Predictive Control under Intermittent Observations and Unreliable Actions](https://mlsanity.com/paper/1911.03617/) (2019)
- [Anytime Control under Practical Communication Model](https://mlsanity.com/paper/2012.00962/) (2020)

## Built on by (most cited first, 5 of 5)

- [Deep Learning for Wireless Networked Systems: a joint Estimation-Control-Scheduling Approach](https://mlsanity.com/paper/2210.00673/) (2022)
- [Structure-Enhanced DRL for Optimal Transmission Scheduling](https://mlsanity.com/paper/2212.12704/) (2022)
- [Structure-Enhanced Deep Reinforcement Learning for Optimal Transmission Scheduling](https://mlsanity.com/paper/2211.10827/) (2022)
- [Semantic-aware Transmission Scheduling: a Monotonicity-driven Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2305.13706/) (2023)
- [Towards Ultra-Reliable 6G in-X Subnetworks: Dynamic Link Adaptation by Deep Reinforcement Learning](https://mlsanity.com/paper/2507.12031/) (2025)

## Abstract (arXiv)

We consider a joint uplink and downlink scheduling problem of a fully distributed wireless networked control system (WNCS) with a limited number of frequency channels. Using elements of stochastic systems theory, we derive a sufficient stability condition of the WNCS, which is stated in terms of both the control and communication system parameters. Once the condition is satisfied, there exists a stationary and deterministic scheduling policy that can stabilize all plants of the WNCS. By analyzing and representing the per-step cost function of the WNCS in terms of a finite-length countable vector state, we formulate the optimal transmission scheduling problem into a Markov decision process and develop a deep reinforcement learning (DRL) based framework for solving it. To tackle the challenges of a large action space in DRL, we propose novel action space reduction and action embedding methods for the DRL framework that can be applied to various algorithms, including Deep Q-Network (DQN), Deep Deterministic Policy Gradient (DDPG), and Twin Delayed Deep Deterministic Policy Gradient (TD3). Numerical results show that the proposed algorithm significantly outperforms benchmark policies.