# Goal-oriented Transmission Scheduling: Structure-guided DRL with a Unified Dual On-policy and Off-policy Approach

arXiv: https://arxiv.org/abs/2501.11921 · HTML: https://mlsanity.com/paper/2501.11921/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 35 references that are in the map.
- First posted on arXiv 21 Jan 2025.

## Builds on (most cited first, 16 of 16)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications](https://mlsanity.com/paper/2207.09353/) (2022)
- [Breaking the Deadly Triad with a Target Network](https://mlsanity.com/paper/2101.08862/) (2021)
- [Deep Reinforcement Learning for Wireless Sensor Scheduling in Cyber-Physical Systems](https://mlsanity.com/paper/1809.05149/) (2018)
- [Remote State Estimation with Smart Sensors over Markov Fading Channels](https://mlsanity.com/paper/2005.07871/) (2020)
- [DRL-based Resource Allocation in Remote State Estimation](https://mlsanity.com/paper/2205.12267/) (2022)
- [Structure-Enhanced DRL for Optimal Transmission Scheduling](https://mlsanity.com/paper/2212.12704/) (2022)
- [Age-Based Scheduling for Mobile Edge Computing: A Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2312.00279/) (2023)
- [Optimal Downlink-Uplink Scheduling of Wireless Networked Control for Industrial IoT](https://mlsanity.com/paper/1907.07841/) (2019)
- [Optimal Scheduling of Multiple Sensors over Lossy and Bandwidth Limited Channels](https://mlsanity.com/paper/1804.05618/) (2018)
- [Stability Conditions for Remote State Estimation of Multiple Systems over Multiple Markov Fading Channels](https://mlsanity.com/paper/2104.04181/) (2021)
- [Goal-Oriented Scheduling in Sensor Networks with Application Timing Awareness](https://mlsanity.com/paper/2306.03750/) (2023)
- [Monitoring Correlated Sources: AoI-based Scheduling is Nearly Optimal](https://mlsanity.com/paper/2312.16813/) (2023)
- [Age of Incorrect Information in Semantic Communications for NOMA Aided XR Applications](https://mlsanity.com/paper/2305.09272/) (2023)
- [Learning Optimal Scheduling Policy for Remote State Estimation under Uncertain Channel Condition](https://mlsanity.com/paper/1810.09820/) (2018)
- [Semantic-aware Transmission Scheduling: a Monotonicity-driven Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2305.13706/) (2023)

## Abstract (arXiv)

Goal-oriented communications prioritize application-driven objectives over data accuracy, enabling intelligent next-generation wireless systems. Efficient scheduling in multi-device, multi-channel systems poses significant challenges due to high-dimensional state and action spaces. We address these challenges by deriving key structural properties of the optimal solution to the goal-oriented scheduling problem, incorporating Age of Information (AoI) and channel states. Specifically, we establish the monotonicity of the optimal state value function (a measure of long-term system performance) w.r.t. channel states and prove its asymptotic convexity w.r.t. AoI states. Additionally, we derive the monotonicity of the optimal policy w.r.t. channel states, advancing the theoretical framework for optimal scheduling. Leveraging these insights, we propose the structure-guided unified dual on-off policy DRL (SUDO-DRL), a hybrid algorithm that combines the stability of on-policy training with the sample efficiency of off-policy methods. Through a novel structural property evaluation framework, SUDO-DRL enables effective and scalable training, addressing the complexities of large-scale systems. Numerical results show SUDO-DRL improves system performance by up to 45% and reduces convergence time by 40% compared to state-of-the-art methods. It also effectively handles scheduling in much larger systems, where off-policy DRL fails and on-policy benchmarks exhibit significant performance loss, demonstrating its scalability and efficacy in goal-oriented communications.