mlsanityData to 7 Oct 2026

Monte Carlo Tree Search with Boltzmann Exploration

arXiv 2404.07732 ↗First posted 11 Apr 2024cs.AI cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

Monte-Carlo Tree Search (MCTS) methods, such as Upper Confidence Bound applied to Trees (UCT), are instrumental to automated planning techniques. However, UCT can be slow to explore an optimal action when it initially appears inferior to other actions. Maximum ENtropy Tree-Search (MENTS) incorporates the maximum entropy principle into an MCTS approach, utilising Boltzmann policies to sample actions, naturally encouraging more exploration. In this paper, we highlight a major limitation of MENTS: optimal actions for the maximum entropy objective do not necessarily correspond to optimal actions for the original objective. We introduce two algorithms, Boltzmann Tree Search (BTS) and Decaying ENtropy Tree-Search (DENTS), that address these limitations and preserve the benefits of Boltzmann policies, such as allowing actions to be sampled faster by using the Alias method. Our empirical analysis shows that our algorithms show consistent high performance across several benchmark domains, including the game of Go.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 9 of its 37 references that are in the map. First posted on arXiv 11 Apr 2024.

Builds on

9 of 37 references.

References · 9

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
  3. 2017Bridging the Gap Between Value and Policy Based Reinforcement Learningcited by 178
  4. 2019Accelerating Self-Play Learning in Gocited by 60
  5. 2014Bandit Algorithms for Tree Searchcited by 25
  6. 2013Credal Networks under Maximum Entropycited by 13
  7. 2020Convex Regularization in Monte-Carlo Tree Searchcited by 8
  8. 2021Planning and Learning Using Adaptive Entropy Tree Searchcited by 3
  9. 2013Monte-Carlo Planning: Theoretically Fast Convergence Meets Practical Efficiencycited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 7; 2026: 4.

12 papers.

  1. 2025SE-Agent: Self-Evolution Trajectory Optimization in Multi-Step Reasoning with LLM-Based Agentscited by 66
  2. 2025Diffusion Tree Sampling: Scalable inference-time alignment of diffusion modelscited by 31
  3. 2025MALinZero: Efficient Low-Dimensional Search for Mastering Complex Multi-Agent Planningcited by 5
  4. 2025Generative Actor Criticcited by 2
  5. 2025Exploration by Random Reward Perturbationcited by 1
  6. 2026Power Mean Estimation in Stochastic Continuous Monte Carlo Tree Searchcited by 1
  7. 2024Improved Monte Carlo Planning via Causal Disentanglement for Structurally-Decomposed Markov Decision Processescited by 0
  8. 2025Doubly Robust Monte Carlo Tree Searchcited by 0
  9. 2025Neural Architecture Search by Learning a Hierarchical Search Spacecited by 0
  10. 2026Online Navigation Planning for Long-term Autonomous Operation of Underwater Gliderscited by 0
  11. 2026Boltzmann-based Exploration for Robust Decentralized Multi-Agent Planning (Extended Version)cited by 0
  12. 2026Variance Driven Exploration: A Provable and Efficient Methodology for Pure Exploration in Highly Stochastic Environmentscited by 0