# A Constrained RL Approach for Cost-Efficient Delivery of Latency-Sensitive Applications

arXiv: https://arxiv.org/abs/2603.04353 · HTML: https://mlsanity.com/paper/2603.04353/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 13 of its 22 references that are in the map.
- First posted on arXiv 4 Mar 2026.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO](https://mlsanity.com/paper/2005.12729/) (2020)
- [Constrained Reinforcement Learning Has Zero Duality Gap](https://mlsanity.com/paper/1910.13393/) (2019)
- [Model-free Reinforcement Learning in Infinite-horizon Average-reward Markov Decision Processes](https://mlsanity.com/paper/1910.07072/) (2019)
- [A Deep-Reinforcement Learning Approach for Software-Defined Networking Routing Optimization](https://mlsanity.com/paper/1709.07080/) (2017)
- [RL-QN: A Reinforcement Learning Framework for Optimal Control of Queueing Systems](https://mlsanity.com/paper/2011.07401/) (2020)
- [Compute- and Data-Intensive Networks: The Key to the Metaverse](https://mlsanity.com/paper/2204.02001/) (2022)
- [Throughput Optimal Decentralized Scheduling of Multi-Hop Networks with End-to-End Deadline Constraints: Unreliable Links](https://mlsanity.com/paper/1606.01608/) (2016)
- [Constraint-Aware Deep Reinforcement Learning for End-to-End Resource Orchestration in Mobile Networks](https://mlsanity.com/paper/2110.04320/) (2021)
- [Optimal Control for Generalized Network-Flow Problems](https://mlsanity.com/paper/1611.08641/) (2016)
- [Ultra-Reliable Distributed Cloud Network Control with End-to-End Latency Constraints](https://mlsanity.com/paper/2205.02427/) (2022)
- [Optimal Control of Distributed Computing Networks with Mixed-Cast Traffic Flows](https://mlsanity.com/paper/1805.10527/) (2018)

## Abstract (arXiv)

Next-generation networks aim to provide performance guarantees to real-time interactive services that require timely and cost-efficient packet delivery. In this context, the goal is to reliably deliver packets with strict deadlines imposed by the application while minimizing overall resource allocation cost. A large body of work has leveraged stochastic optimization techniques to design efficient dynamic routing and scheduling solutions under average delay constraints; however, these methods fall short when faced with strict per-packet delay requirements. We formulate the minimum-cost delay-constrained network control problem as a constrained Markov decision process and utilize constrained deep reinforcement learning (CDRL) techniques to effectively minimize total resource allocation cost while maintaining timely throughput above a target reliability level. Results indicate that the proposed CDRL-based solution can ensure timely packet delivery even when existing baselines fall short, and it achieves lower cost compared to other throughput-maximizing methods.