mlsanityData to 14 Oct 2026

C-MCTS: Safe Planning with Monte Carlo Tree Search

arXiv 2305.16209 ↗First posted 25 May 2023cs.LG cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

The Constrained Markov Decision Process (CMDP) formulation allows to solve safety-critical decision making tasks that are subject to constraints. While CMDPs have been extensively studied in the Reinforcement Learning literature, little attention has been given to sampling-based planning algorithms such as MCTS for solving them. Previous approaches perform conservatively with respect to costs as they avoid constraint violations by using Monte Carlo cost estimates that suffer from high variance. We propose Constrained MCTS (C-MCTS), which estimates cost using a safety critic that is trained with Temporal Difference learning in an offline phase prior to agent deployment. The critic limits exploration by pruning unsafe trajectories within MCTS during deployment. C-MCTS satisfies cost constraints but operates closer to the constraint boundary, achieving higher rewards than previous work. As a nice byproduct, the planner is more efficient w.r.t. planning steps. Most importantly, under model mismatch between the planner and the real world, C-MCTS is less susceptible to cost violations than previous work.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 22 of its 43 references that are in the map. First posted on arXiv 25 May 2023.

Builds on

22 of 43 references.

References · 22

  1. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  2. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 950
  3. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
  4. 2017Deep Reinforcement Learning that Matterscited by 754
  5. 2018Reward Constrained Policy Optimizationcited by 417
  6. 2021Reinforcement learning based recommender systems: A surveycited by 188
  7. 2021Monte Carlo Tree Search: A Review of Recent Modifications and Applicationscited by 161
  8. 2020Learning to be Safe: Deep RL with a Safety Criticcited by 116
  9. 2020Conservative Safety Critics for Explorationcited by 113
  10. 2017Cautious Model Predictive Control using Gaussian Process Regressioncited by 91
  11. 2022When Should We Prefer Offline Reinforcement Learning Over Behavioral Cloning?cited by 89
  12. 2023Constrained Decision Transformer for Offline Safe Reinforcement Learningcited by 69
10 more
  1. 2019Combining Planning and Deep Reinforcement Learning in Tactical Decision Making for Autonomous Drivingcited by 52
  2. 2022Model-based Safe Deep Reinforcement Learning via a Constrained Proximal Policy Optimization Algorithmcited by 29
  3. 2020Robust Constrained Reinforcement Learning for Continuous Control with Model Misspecificationcited by 16
  4. 2022On the Convergence of SARSA with Linear Function Approximationcited by 15
  5. 2017Probabilistic Motion Planning under Temporal Tasks and Soft Constraintscited by 9
  6. 2023Choose Your Weapon: Survival Strategies for Depressed AI Academicscited by 8
  7. 2021Pareto Monte Carlo Tree Search for Multi-Objective Informative Planningcited by 7
  8. 2021Reinforcement Learning Based Safe Decision Making for Highway Autonomous Drivingcited by 5
  9. 2022Optimal Control of Partially Observable Markov Decision Processes with Finite Linear Temporal Logic Constraintscited by 5
  10. 2022Monte Carlo Tree Search Algorithms for Risk-Aware and Multi-Objective Reinforcement Learningcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 2; 2026: 1.

6 papers.

  1. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 15
  2. 2024Mirror: A Multiple-perspective Self-Reflection Method for Knowledge-rich Reasoningcited by 10
  3. 2024ConstrainedZero: Chance-Constrained POMDP Planning using Learned Probabilistic Failure Surrogates and Adaptive Safety Constraintscited by 8
  4. 2025Tail-Risk-Safe Monte Carlo Tree Search under PAC-Level Guaranteescited by 4
  5. 2026Online Robust Reinforcement Learning Through Monte-Carlo Planningcited by 2
  6. 2023Constrained Hierarchical Monte Carlo Belief-State Planningcited by 1