# Resource Constrained Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1812.00600 · HTML: https://mlsanity.com/paper/1812.00600/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Dec 2018.

## Built on by (most cited first, 24 of 24)

- [Escaping from Zero Gradient: Revisiting Action-Constrained Reinforcement Learning via Frank-Wolfe Policy Optimization](https://mlsanity.com/paper/2102.11055/) (2021)
- [FlowPG: Action-constrained Policy Gradient with Normalizing Flows](https://mlsanity.com/paper/2402.05149/) (2024)
- [Benchmarking Actor-Critic Deep Reinforcement Learning Algorithms for Robotics Control with Action Constraints](https://mlsanity.com/paper/2304.08743/) (2023)
- [Efficient Exploration in Resource-Restricted Reinforcement Learning](https://mlsanity.com/paper/2212.06988/) (2022)
- [Generative Modelling of Stochastic Actions with Arbitrary Constraints in Reinforcement Learning](https://mlsanity.com/paper/2311.15341/) (2023)
- [Efficient Action-Constrained Reinforcement Learning via Acceptance-Rejection Method and Augmented MDPs](https://mlsanity.com/paper/2503.12932/) (2025)
- [CLARA: A Constrained Reinforcement Learning Based Resource Allocation Framework for Network Slicing](https://mlsanity.com/paper/2111.08397/) (2021)
- [GLinSAT: The General Linear Satisfiability Neural Network Layer By Accelerated Gradient Descent](https://mlsanity.com/paper/2409.17500/) (2024)
- [Reduced Policy Optimization for Continuous Control with Hard Constraints](https://mlsanity.com/paper/2310.09574/) (2023)
- [Anytime-Constrained Reinforcement Learning](https://mlsanity.com/paper/2311.05511/) (2023)
- [Deterministic Policies for Constrained Reinforcement Learning in Polynomial Time](https://mlsanity.com/paper/2405.14183/) (2024)
- [Safe Reinforcement Learning using Action Projection: Safeguard the Policy or the Environment?](https://mlsanity.com/paper/2509.12833/) (2025)
- [Leveraging Constraint Violation Signals For Action-Constrained Reinforcement Learning](https://mlsanity.com/paper/2502.10431/) (2025)
- [Action-Constrained Imitation Learning](https://mlsanity.com/paper/2508.14379/) (2025)
- [Joint Continual Learning of Local Language Models and Cloud Offloading Decisions with Budget Constraints](https://mlsanity.com/paper/2602.00166/) (2026)
- [Constrained Reinforcement Learning for Dynamic Material Handling](https://mlsanity.com/paper/2305.13824/) (2023)
- [Anytime-Constrained Equilibria in Polynomial Time](https://mlsanity.com/paper/2410.23637/) (2024)
- [Polynomial-Time Approximability of Constrained Reinforcement Learning](https://mlsanity.com/paper/2502.07764/) (2025)
- [Neural-Progressive Hedging: Enforcing Constraints in Reinforcement Learning with Stochastic Programming](https://mlsanity.com/paper/2202.13436/) (2022)
- [mathrmE2CFD: Towards Effective and Efficient Cost Function Design for Safe Reinforcement Learning via Large Language Model](https://mlsanity.com/paper/2407.05580/) (2024)
- [Autoregressive Policy Optimization for Constrained Allocation Tasks](https://mlsanity.com/paper/2409.18735/) (2024)
- [Navigating Demand Uncertainty in Container Shipping: Deep Reinforcement Learning for Enabling Adaptive and Feasible Master Stowage Planning](https://mlsanity.com/paper/2502.12756/) (2025)
- [Situational-Constrained Sequential Resources Allocation via Reinforcement Learning](https://mlsanity.com/paper/2506.14125/) (2025)
- [Neural Co-state Projection Regulator: A Model-free Paradigm for Real-time Optimal Control with Input Constraints](https://mlsanity.com/paper/2508.00283/) (2025)

## Abstract (arXiv)

In urban environments, supply resources have to be constantly matched to the "right" locations (where customer demand is present) so as to improve quality of life. For instance, ambulances have to be matched to base stations regularly so as to reduce response time for emergency incidents in EMS (Emergency Management Systems); vehicles (cars, bikes, scooters etc.) have to be matched to docking stations so as to reduce lost demand in shared mobility systems. Such problem domains are challenging owing to the demand uncertainty, combinatorial action spaces (due to allocation) and constraints on allocation of resources (e.g., total resources, minimum and maximum number of resources at locations and regions). Existing systems typically employ myopic and greedy optimization approaches to optimize allocation of supply resources to locations. Such approaches typically are unable to handle surges or variances in demand patterns well. Recent research has demonstrated the ability of Deep RL methods in adapting well to highly uncertain environments. However, existing Deep RL methods are unable to handle combinatorial action spaces and constraints on allocation of resources. To that end, we have developed three approaches on top of the well known actor critic approach, DDPG (Deep Deterministic Policy Gradient) that are able to handle constraints on resource allocation. More importantly, we demonstrate that they are able to outperform leading approaches on simulators validated on semi-real and real data sets.