mlsanityData to 14 Oct 2026

Safe Reinforcement Learning for Constrained Markov Decision Processes with Stochastic Stopping Time

arXiv 2403.15928 ↗First posted 23 Mar 2024cs.LG math.OCcited by 5

On the map: Deep reinforcement learning methods → Safe and constrained RL

Abstract (arXiv)

In this paper, we present an online reinforcement learning algorithm for constrained Markov decision processes with a safety constraint. Despite the necessary attention of the scientific community, considering stochastic stopping time, the problem of learning optimal policy without violating safety constraints during the learning phase is yet to be addressed. To this end, we propose an algorithm based on linear programming that does not require a process model. We show that the learned policy is safe with high confidence. We also propose a method to compute a safe baseline policy, which is central in developing algorithms that do not violate the safety constraints. Finally, we provide simulation results to show the efficacy of the proposed algorithm. Further, we demonstrate that efficient exploration can be achieved by defining a subset of the state-space called proxy set.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. Builds on 9 of its 26 references that are in the map. First posted on arXiv 23 Mar 2024.

Builds on

9 of 26 references.

References · 9

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  2. 2020Exploration-Exploitation in Constrained MDPscited by 149
  3. 2020Safe Reinforcement Learning in Constrained Markov Decision Processescited by 115
  4. 2019Convergent Policy Optimization for Safe Reinforcement Learningcited by 57
  5. 2021DOPE: Doubly Optimistic and Pessimistic Exploration for Safe Reinforcement Learningcited by 35
  6. 2019Smoothing Policies and Safe Policy Gradientscited by 21
  7. 2023Provably Efficient Model-Free Algorithms for Non-stationary CMDPscited by 21
  8. 2020Learning in Markov Decision Processes under Constraintscited by 20
  9. 2022A policy gradient approach for Finite Horizon Constrained Markov Decision Processescited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 3.

5 papers.

  1. 2026BAGEN: Are LLM Agents Budget-Aware?cited by 13
  2. 2025Probabilistic Shielding for Safe Reinforcement Learningcited by 12
  3. 2024SaVeR: Optimal Data Collection Strategy for Safe Policy Evaluation in Tabular MDPcited by 3
  4. 2026Provably Safe Reinforcement Learning for Stochastic Reach-Avoid Problems with Entropy Regularizationcited by 0
  5. 2026Planning or Learning: Reliability and Cost in Multi-Asset Maintenancecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Near-Optimal Sample Complexity for Online Constrained MDPs
  2. 2025A Survey of Safe Reinforcement Learning and Constrained MDPs: A Technical Survey on Single-Agent and Multi-Agent Safety
  3. 2024Flipping-based Policy for Chance-Constrained Markov Decision Processes
  4. 2025Ensuring Safety in an Uncertain Environment: Constrained MDPs via Stochastic Thresholds
  5. 2024Safe Reinforcement Learning with Learned Non-Markovian Safety Constraints
  6. 2025Provably Optimal Reinforcement Learning under Safety Filtering
  7. 2024A Survey of Constraint Formulations in Safe Reinforcement Learning
  8. 2021Constraints Penalized Q-learning for Safe Offline Reinforcement Learning