# Static and Dynamic Values of Computation in MCTS

arXiv: https://arxiv.org/abs/2002.04335 · HTML: https://mlsanity.com/paper/2002.04335/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Feb 2020.

## Built on by (most cited first, 5 of 5)

- [Improving Human Decision-Making by Discovering Efficient Strategies for Hierarchical Planning](https://mlsanity.com/paper/2102.00521/) (2021)
- [Finding the Time to Think: Learning Planning Budgets in Real-Time RL](https://mlsanity.com/paper/2606.26463/) (2026)
- [Deep imagination is a close to optimal policy for planning in large decision trees under limited resources](https://mlsanity.com/paper/2104.06339/) (2021)
- [Certified-Gap Dual-Price Policies for Real-Time Truckload Bid Acceptance with Relocating, Clock-Constrained Resources](https://mlsanity.com/paper/2607.16891/) (2026)
- [Search as Computation Allocation](https://mlsanity.com/paper/2607.27871/) (2026)

## Abstract (arXiv)

Monte-Carlo Tree Search (MCTS) is one of the most-widely used methods for planning, and has powered many recent advances in artificial intelligence. In MCTS, one typically performs computations (i.e., simulations) to collect statistics about the possible future consequences of actions, and then chooses accordingly. Many popular MCTS methods such as UCT and its variants decide which computations to perform by trading-off exploration and exploitation. In this work, we take a more direct approach, and explicitly quantify the value of a computation based on its expected impact on the quality of the action eventually chosen. Our approach goes beyond the "myopic" limitations of existing computation-value-based methods in two senses: (I) we are able to account for the impact of non-immediate (ie, future) computations (II) on non-immediate actions. We show that policies that greedily optimize computation values are optimal under certain assumptions and obtain results that are competitive with the state-of-the-art.