Is Monte Carlo Tree Search Just Every-Visit Monte Carlo Control?
Not on the map. Why?
Abstract (arXiv)
Monte Carlo Tree Search (MCTS) and every-visit Monte Carlo (MC) control are usually presented as different methods. MCTS is described in the language of search (selection, expansion, simulation, and backup), whereas MC control is described in the language of reinforcement learning (trajectory sampling, return estimation, action-value updating, and policy improvement). This note argues that, at the level of trajectory generation and action-value updating, the distinction is largely terminological. The tree policy and rollout policy can be viewed as the learned and not-yet-learned parts of a single evolving policy; expansion corresponds to first visit and initialization; and backup is the ordinary every-visit Monte Carlo update. Under this interpretation, the four stages of MCTS reduce to two basic operations: trajectory sampling under the current policy and every-visit Monte Carlo updating. In this sense, MCTS is simply every-visit Monte Carlo control expressed in the language and data structure of search. The purpose of this note is expository: to make this equivalence explicit and easier to recognize.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 0 of its 6 references that are in the map. First posted on arXiv 28 Aug 2026.
Builds on
0 of 6 references.
No references here yet.
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Variance-Aware Prior-Based Tree Policies for Monte Carlo Tree Search
- 2025Twice Sequential Monte Carlo for Tree Search
- 2024Optimized Monte Carlo Tree Search for Enhanced Decision Making in the FrozenLake Environment
- 2026Particle Monte Carlo Tree Search
- 2024Decision Making in Non-Stationary Environments with Policy-Augmented Search
- 2026Online Robust Reinforcement Learning Through Monte-Carlo Planning
- 2025Monte Carlo Permutation Search
- 2025Trust-Region Twisted Policy Improvement