mlsanityData to 7 Oct 2026

Constrained Upper Confidence Reinforcement Learning

arXiv 2001.09377 ↗First posted 26 Jan 2020cs.LG stat.MLcited by 46

Not on the map. Why?

Abstract (arXiv)

Constrained Markov Decision Processes are a class of stochastic decision problems in which the decision maker must select a policy that satisfies auxiliary cost constraints. This paper extends upper confidence reinforcement learning for settings in which the reward function and the constraints, described by cost functions, are unknown a priori but the transition kernel is known. Such a setting is well-motivated by a number of applications including exploration of unknown, potentially unsafe, environments. We present an algorithm C-UCRL and show that it achieves sub-linear regret ( O(T^frac34√log(T/δ))) with respect to the reward while satisfying the constraints even while learning with probability 1-δ. Illustrative examples are provided.

Key facts. Cited by 46 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 Jan 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 7; 2023: 10; 2024: 8; 2025: 7; 2026: 5.

46 papers.

Sort:

By area:

  1. 2021Recent Advances in Reinforcement Learning in Financecited by 102
  2. 2021Learning Policies with Zero or Bounded Constraint Violation for Constrained MDPscited by 78
  3. 2022Near-Optimal Sample Complexity Bounds for Constrained MDPscited by 36
  4. 2021DOPE: Doubly Optimistic and Pessimistic Exploration for Safe Reinforcement Learningcited by 34
  5. 2021Safe Reinforcement Learning with Linear Function Approximationcited by 33
  6. 2022Learning Infinite-Horizon Average-Reward Markov Decision Processes with Constraintscited by 29
  7. 2022Safe Reinforcement Learning for Legged Locomotioncited by 29
  8. 2022Provably Efficient Model-Free Constrained RL with Linear Function Approximationcited by 29
  9. 2023Last-Iterate Convergent Policy Gradient Primal-Dual Methods for Constrained MDPscited by 27
  10. 2022Convergence and sample complexity of natural policy gradient primal-dual methods for constrained MDPscited by 26
  11. 2023Long-Term Fairness with Unknown Dynamicscited by 25
  12. 2024ActSafe: Active Exploration with Safety Constraints for Reinforcement Learningcited by 22
34 more
  1. 2021Concave Utility Reinforcement Learning with Zero-Constraint Violationscited by 18
  2. 2023Provably Efficient Generalized Lagrangian Policy Optimization for Safe Multi-Agent Reinforcement Learningcited by 13
  3. 2026BAGEN: Are LLM Agents Budget-Aware?cited by 12
  4. 2023A safe exploration approach to constrained Markov decision processescited by 10
  5. 2024Near-Optimal Policy Identification in Robust Constrained Markov Decision Processes via Epigraph Formcited by 10
  6. 2022Safe Exploration Incurs Nearly No Additional Sample Complexity for Reward-free RLcited by 9
  7. 2024Markov Persuasion Processes: Learning to Persuade from Scratchcited by 8
  8. 2021Safe Value Functionscited by 7
  9. 2023Safe Posterior Sampling for Constrained MDPs with Bounded Constraint Violationcited by 6
  10. 2024Constrained Reinforcement Learning Under Model Mismatchcited by 6
  11. 2025Efficient Policy Optimization in Robust Constrained MDPs with Iteration Complexity Guaranteescited by 6
  12. 2024A Policy Gradient Primal-Dual Algorithm for Constrained MDPs with Uniform PAC Guaranteescited by 5
  13. 2023Near-optimal Conservative Exploration in Reinforcement Learning under Episode-wise Constraintscited by 4
  14. 2024Confident Natural Policy Gradient for Local Planning in qπ-realizable Constrained MDPscited by 4
  15. 2021Markov Decision Processes with Long-Term Average Constraintscited by 3
  16. 2023Provable Reset-free Reinforcement Learning by No-Regret Reductioncited by 3
  17. 2025Near-Optimal Sample Complexity Bounds for Constrained Average-Reward MDPscited by 3
  18. 2021Online Selection of Diverse Committeescited by 2
  19. 2024Efficient Exploration in Average-Reward Constrained Reinforcement Learning: Achieving Near-Optimal Regret With Posterior Samplingcited by 2
  20. 2025Data-Dependent Regret Bounds for Constrained MABscited by 2
  21. 2026Replicable Constrained Banditscited by 2
  22. 2021Constraint Sampling Reinforcement Learning: Incorporating Expertise For Faster Learningcited by 1
  23. 2022An Empirical Evaluation of Posterior Sampling for Constrained Reinforcement Learningcited by 1
  24. 2024A Primal-Dual Online Learning Approach for Dynamic Pricing of Sequentially Displayed Complementary Items under Sale Constraintscited by 1
  25. 2025Beyond Slater's Condition in Online CMDPs with Stochastic and Adversarial Constraintscited by 1
  26. 2025Exchange Policy Optimization Algorithm for Semi-Infinite Safe Reinforcement Learningcited by 1
  27. 2025Provably Efficient Sample Complexity for Robust CMDPcited by 1
  28. 2023Semi-Infinitely Constrained Markov Decision Processes and Efficient Reinforcement Learningcited by 0
  29. 2023Model-Free, Regret-Optimal Best Policy Identification in Online CMDPscited by 0
  30. 2023Provably Efficient Exploration in Constrained Reinforcement Learning:Posterior Sampling Is All You Needcited by 0
  31. 2025A Domain-Based Taxonomy of Jailbreak Vulnerabilities in Large Language Modelscited by 0
  32. 2026Truly Adapting to Adversarial Constraints in Constrained MABscited by 0
  33. 2026Offline Constrained RLHF with Multiple Preference Oraclescited by 0
  34. 2026Robust Peak-cost Constrained Reinforcement Learningcited by 0