# Deep Learning for Wireless Networked Systems: a joint Estimation-Control-Scheduling Approach

arXiv: https://arxiv.org/abs/2210.00673 · HTML: https://mlsanity.com/paper/2210.00673/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 49 references that are in the map.
- First posted on arXiv 3 Oct 2022.

## Builds on (most cited first, 21 of 21)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [Age of Information: An Introduction and Survey](https://mlsanity.com/paper/2007.08564/) (2020)
- [Semantics-Empowered Communication for Networked Intelligent Systems](https://mlsanity.com/paper/2007.11579/) (2020)
- [Memory-based Deep Reinforcement Learning for POMDPs](https://mlsanity.com/paper/2102.12344/) (2021)
- [Semantic Communications in Networked Systems: A Data Significance Perspective](https://mlsanity.com/paper/2103.05391/) (2021)
- [Deep Reinforcement Learning for Wireless Sensor Scheduling in Cyber-Physical Systems](https://mlsanity.com/paper/1809.05149/) (2018)
- [Deep Reinforcement Learning for Event-Triggered Control](https://mlsanity.com/paper/1809.05152/) (2018)
- [Wireless Network Design for Control Systems: A Survey](https://mlsanity.com/paper/1708.07353/) (2017)
- [Wireless Access in Ultra-Reliable Low-Latency Communication (URLLC)](https://mlsanity.com/paper/1810.06938/) (2018)
- [Control Aware Radio Resource Allocation in Low Latency Wireless Control Systems](https://mlsanity.com/paper/1811.00409/) (2018)
- [Model-Free Design of Control Systems over Wireless Fading Channels](https://mlsanity.com/paper/2009.01751/) (2020)
- [Optimal Sensor Scheduling for Multiple Linear Dynamical Systems](https://mlsanity.com/paper/1604.04158/) (2016)
- [Learning to Represent Haptic Feedback for Partially-Observable Tasks](https://mlsanity.com/paper/1705.06243/) (2017)
- [On the Latency, Rate and Reliability Tradeoff in Wireless Networked Control Systems for IIoT](https://mlsanity.com/paper/2007.00256/) (2020)
- [Deep Reinforcement Learning for Wireless Scheduling in Distributed Networked Control](https://mlsanity.com/paper/2109.12562/) (2021)
- [Wireless Networked Control Systems with Coding-Free Data Transmission for Industrial IoT](https://mlsanity.com/paper/1907.13297/) (2019)
- [Wireless Feedback Control with Variable Packet Length for Industrial IoT](https://mlsanity.com/paper/2005.13321/) (2020)
- [On the Optimal Solutions of the Infinite-Horizon Linear Sensor Scheduling Problem](https://mlsanity.com/paper/1312.0157/) (2013)
- [Sensor Scheduling in Variance Based Event Triggered Estimation with Packet Drops](https://mlsanity.com/paper/1511.04792/) (2015)

## Built on by (most cited first, 7 of 7)

- [Time-Series JEPA for Predictive Remote Control under Capacity-Limited Networks](https://mlsanity.com/paper/2406.04853/) (2024)
- [Diffusion Model Based Resource Allocation Strategy in Ultra-Reliable Wireless Networked Control Systems](https://mlsanity.com/paper/2407.15784/) (2024)
- [Structure-Enhanced DRL for Optimal Transmission Scheduling](https://mlsanity.com/paper/2212.12704/) (2022)
- [Optimization Theory Based Deep Reinforcement Learning for Resource Allocation in Ultra-Reliable Wireless Networked Control Systems](https://mlsanity.com/paper/2311.16895/) (2023)
- [Structure-Enhanced Deep Reinforcement Learning for Optimal Transmission Scheduling](https://mlsanity.com/paper/2211.10827/) (2022)
- [Safe Deep Reinforcement Learning for Resource Allocation with Peak Age of Information Violation Guarantees](https://mlsanity.com/paper/2507.08653/) (2025)
- [Communication-Control Codesign for Large-Scale Wireless Networked Control Systems](https://mlsanity.com/paper/2410.11316/) (2024)

## Abstract (arXiv)

Wireless networked control system (WNCS) connecting sensors, controllers, and actuators via wireless communications is a key enabling technology for highly scalable and low-cost deployment of control systems in the Industry 4.0 era. Despite the tight interaction of control and communications in WNCSs, most existing works adopt separative design approaches. This is mainly because the co-design of control-communication policies requires large and hybrid state and action spaces, making the optimal problem mathematically intractable and difficult to be solved effectively by classic algorithms. In this paper, we systematically investigate deep learning (DL)-based estimator-control-scheduler co-design for a model-unknown nonlinear WNCS over wireless fading channels. In particular, we propose a co-design framework with the awareness of the sensor's age-of-information (AoI) states and dynamic channel states. We propose a novel deep reinforcement learning (DRL)-based algorithm for controller and scheduler optimization utilizing both model-free and model-based data. An AoI-based importance sampling algorithm that takes into account the data accuracy is proposed for enhancing learning efficiency. We also develop novel schemes for enhancing the stability of joint training. Extensive experiments demonstrate that the proposed joint training algorithm can effectively solve the estimation-control-scheduling co-design problem in various scenarios and provide significant performance gain compared to separative design and some benchmark policies.