# Structure-Enhanced Deep Reinforcement Learning for Optimal Transmission Scheduling

arXiv: https://arxiv.org/abs/2211.10827 · HTML: https://mlsanity.com/paper/2211.10827/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 18 references that are in the map.
- First posted on arXiv 20 Nov 2022.

## Builds on (most cited first, 13 of 13)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Deep Reinforcement Learning for Wireless Sensor Scheduling in Cyber-Physical Systems](https://mlsanity.com/paper/1809.05149/) (2018)
- [Wireless Network Design for Control Systems: A Survey](https://mlsanity.com/paper/1708.07353/) (2017)
- [Directed Exploration for Reinforcement Learning](https://mlsanity.com/paper/1906.07805/) (2019)
- [Remote State Estimation with Smart Sensors over Markov Fading Channels](https://mlsanity.com/paper/2005.07871/) (2020)
- [DRL-based Resource Allocation in Remote State Estimation](https://mlsanity.com/paper/2205.12267/) (2022)
- [Deep Learning for Wireless Networked Systems: a joint Estimation-Control-Scheduling Approach](https://mlsanity.com/paper/2210.00673/) (2022)
- [Optimal Sensor Scheduling for Multiple Linear Dynamical Systems](https://mlsanity.com/paper/1604.04158/) (2016)
- [Optimal Downlink-Uplink Scheduling of Wireless Networked Control for Industrial IoT](https://mlsanity.com/paper/1907.07841/) (2019)
- [Deep Reinforcement Learning for Wireless Scheduling in Distributed Networked Control](https://mlsanity.com/paper/2109.12562/) (2021)
- [Optimal Scheduling of Multiple Sensors over Lossy and Bandwidth Limited Channels](https://mlsanity.com/paper/1804.05618/) (2018)
- [Stability Conditions for Remote State Estimation of Multiple Systems over Multiple Markov Fading Channels](https://mlsanity.com/paper/2104.04181/) (2021)

## Built on by (most cited first, 1 of 1)

- [Structure-Enhanced DRL for Optimal Transmission Scheduling](https://mlsanity.com/paper/2212.12704/) (2022)

## Abstract (arXiv)

Remote state estimation of large-scale distributed dynamic processes plays an important role in Industry 4.0 applications. In this paper, by leveraging the theoretical results of structural properties of optimal scheduling policies, we develop a structure-enhanced deep reinforcement learning (DRL) framework for optimal scheduling of a multi-sensor remote estimation system to achieve the minimum overall estimation mean-square error (MSE). In particular, we propose a structure-enhanced action selection method, which tends to select actions that obey the policy structure. This explores the action space more effectively and enhances the learning efficiency of DRL agents. Furthermore, we introduce a structure-enhanced loss function to add penalty to actions that do not follow the policy structure. The new loss function guides the DRL to converge to the optimal policy structure quickly. Our numerical results show that the proposed structure-enhanced DRL algorithms can save the training time by 50% and reduce the remote estimation MSE by 10% to 25%, when compared to benchmark DRL algorithms.