mlsanityData to 7 Oct 2026

Provably Efficient Long-Horizon Exploration in Monte Carlo Tree Search through State Occupancy Regularization

arXiv 2407.05511 ↗First posted 7 Jul 2024cs.LG cs.ROcited by 1

Not on the map. Why?

Abstract (arXiv)

Monte Carlo tree search (MCTS) has been successful in a variety of domains, but faces challenges with long-horizon exploration when compared to sampling-based motion planning algorithms like Rapidly-Exploring Random Trees. To address these limitations of MCTS, we derive a tree search algorithm based on policy optimization with state occupancy measure regularization, which we call it Volume-MCTS. We show that count-based exploration and sampling-based motion planning can be derived as approximate solutions to this state occupancy measure regularized objective. We test our method on several robot navigation problems, and find that Volume-MCTS outperforms AlphaZero and displays significantly better long-horizon exploration properties.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 18 of its 29 references that are in the map. First posted on arXiv 7 Jul 2024.

Builds on

18 of 29 references.

References · 18

  1. 2017Hindsight Experience Replaycited by 1,096
  2. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 300
  3. 2018Provably Efficient Maximum Entropy Explorationcited by 240
  4. 2020Never Give Up: Learning Directed Exploration Strategiescited by 210
  5. 2018Count-Based Exploration with the Successor Representationcited by 112
  6. 2021State Entropy Maximization with Random Encoders for Efficient Explorationcited by 107
  7. 2020Monte-Carlo Tree Search as Regularized Policy Optimizationcited by 59
  8. 2019Autonomous Penetration Testing using Reinforcement Learningcited by 33
  9. 2022R'enyi State Entropy for Exploration Acceleration in Reinforcement Learningcited by 18
  10. 2014Asymptotically Optimal Sampling-based Kinodynamic Planningcited by 17
  11. 2018A0C: Alpha Zero in Continuous Action Spacecited by 17
  12. 2018Probabilistic completeness of RRT for geometric and kinodynamic planning with forward propagationcited by 9
6 more
  1. 2015Asymptotically Optimal Planning by Feasible Kinodynamic Planning in State-Cost Spacecited by 8
  2. 2020POLY-HOOT: Monte-Carlo Planning in Continuous Space MDPs with Non-Asymptotic Analysiscited by 8
  3. 2020CARLA Real Traffic Scenarios -- novel training ground and benchmark for autonomous drivingcited by 8
  4. 2022A Survey on the Integration of Machine Learning with Sampling-based Motion Planningcited by 6
  5. 2014A Comparison of Monte Carlo Tree Search and Mathematical Optimization for Large Scale Dynamic Resource Allocationcited by 2
  6. 2023Roadmaps with Gaps over Controllers: Achieving Efficiency in Planning under Dynamicscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Online Decision-Making Under Uncertainty for Vehicle-to-Building Systemscited by 0