# Structure-Enhanced DRL for Optimal Transmission Scheduling

arXiv: https://arxiv.org/abs/2212.12704 · HTML: https://mlsanity.com/paper/2212.12704/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 41 references that are in the map.
- First posted on arXiv 24 Dec 2022.

## Builds on (most cited first, 18 of 18)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Semantic Communications for Future Internet: Fundamentals, Applications, and Challenges](https://mlsanity.com/paper/2207.00427/) (2022)
- [Convolutional Neural Networks at Constrained Time Cost](https://mlsanity.com/paper/1412.1710/) (2014)
- [Deep Reinforcement Learning for Wireless Sensor Scheduling in Cyber-Physical Systems](https://mlsanity.com/paper/1809.05149/) (2018)
- [Wireless Network Design for Control Systems: A Survey](https://mlsanity.com/paper/1708.07353/) (2017)
- [DeepCAS: A Deep Reinforcement Learning Algorithm for Control-Aware Scheduling](https://mlsanity.com/paper/1803.02998/) (2018)
- [Directed Exploration for Reinforcement Learning](https://mlsanity.com/paper/1906.07805/) (2019)
- [Remote State Estimation with Smart Sensors over Markov Fading Channels](https://mlsanity.com/paper/2005.07871/) (2020)
- [DRL-based Resource Allocation in Remote State Estimation](https://mlsanity.com/paper/2205.12267/) (2022)
- [Deep Learning for Wireless Networked Systems: a joint Estimation-Control-Scheduling Approach](https://mlsanity.com/paper/2210.00673/) (2022)
- [Optimal Sensor Scheduling for Multiple Linear Dynamical Systems](https://mlsanity.com/paper/1604.04158/) (2016)
- [Optimal Downlink-Uplink Scheduling of Wireless Networked Control for Industrial IoT](https://mlsanity.com/paper/1907.07841/) (2019)
- [Joint Resource Management for MC-NOMA: A Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2103.15371/) (2021)
- [Deep Reinforcement Learning for Wireless Scheduling in Distributed Networked Control](https://mlsanity.com/paper/2109.12562/) (2021)
- [Optimal Scheduling of Multiple Sensors over Lossy and Bandwidth Limited Channels](https://mlsanity.com/paper/1804.05618/) (2018)
- [Stability Conditions for Remote State Estimation of Multiple Systems over Multiple Markov Fading Channels](https://mlsanity.com/paper/2104.04181/) (2021)
- [Learning Optimal Scheduling Policy for Remote State Estimation under Uncertain Channel Condition](https://mlsanity.com/paper/1810.09820/) (2018)
- [Structure-Enhanced Deep Reinforcement Learning for Optimal Transmission Scheduling](https://mlsanity.com/paper/2211.10827/) (2022)

## Built on by (most cited first, 6 of 6)

- [Improve the Training Efficiency of DRL for Wireless Communication Resource Allocation: The Role of Generative Diffusion Models](https://mlsanity.com/paper/2502.07211/) (2025)
- [Semantic-aware Transmission Scheduling: a Monotonicity-driven Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2305.13706/) (2023)
- [Belief-Conditioned One-Step Diffusion: Real-Time Trajectory Planning with Just-Enough Sensing](https://mlsanity.com/paper/2508.12166/) (2025)
- [Wireless Human-Machine Collaboration in Industry 5.0](https://mlsanity.com/paper/2410.14153/) (2024)
- [Goal-oriented Transmission Scheduling: Structure-guided DRL with a Unified Dual On-policy and Off-policy Approach](https://mlsanity.com/paper/2501.11921/) (2025)
- [Digital Twin-Driven Adaptive Sim-to-Real Alignment via Reinforcement Learning for Vibration-Based Bearing Health Monitoring Under Data Scarcity](https://mlsanity.com/paper/2606.24954/) (2026)

## Abstract (arXiv)

Remote state estimation of large-scale distributed dynamic processes plays an important role in Industry 4.0 applications. In this paper, we focus on the transmission scheduling problem of a remote estimation system. First, we derive some structural properties of the optimal sensor scheduling policy over fading channels. Then, building on these theoretical guidelines, we develop a structure-enhanced deep reinforcement learning (DRL) framework for optimal scheduling of the system to achieve the minimum overall estimation mean-square error (MSE). In particular, we propose a structure-enhanced action selection method, which tends to select actions that obey the policy structure. This explores the action space more effectively and enhances the learning efficiency of DRL agents. Furthermore, we introduce a structure-enhanced loss function to add penalties to actions that do not follow the policy structure. The new loss function guides the DRL to converge to the optimal policy structure quickly. Our numerical experiments illustrate that the proposed structure-enhanced DRL algorithms can save the training time by 50% and reduce the remote estimation MSE by 10% to 25% when compared to benchmark DRL algorithms. In addition, we show that the derived structural properties exist in a wide range of dynamic scheduling problems that go beyond remote state estimation.