# Generalized Mean Estimation in Monte-Carlo Tree Search

arXiv: https://arxiv.org/abs/1911.00384 · HTML: https://mlsanity.com/paper/1911.00384/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Nov 2019.

## Built on by (most cited first, 8 of 8)

- [Optimal Defender Strategies for CAGE-2 using Causal Modeling and Tree Search](https://mlsanity.com/paper/2407.11070/) (2024)
- [A Unified Perspective on Value Backup and Exploration in Monte-Carlo Tree Search](https://mlsanity.com/paper/2202.07071/) (2022)
- [Monte-Carlo tree search with uncertainty propagation via optimal transport](https://mlsanity.com/paper/2309.10737/) (2023)
- [Power Mean Estimation in Stochastic Monte-Carlo Tree_Search](https://mlsanity.com/paper/2406.02235/) (2024)
- [Alpha2: Discovering Logical Formulaic Alphas using Deep Reinforcement Learning](https://mlsanity.com/paper/2406.16505/) (2024)
- [Variance Driven Exploration: A Provable and Efficient Methodology for Pure Exploration in Highly Stochastic Environments](https://mlsanity.com/paper/2608.21995/) (2026)
- [Graph-Based Stochastic Power-UCT: Monte-Carlo Graph Search with Power Mean Estimation](https://mlsanity.com/paper/2609.19956/) (2026)
- [Modular Discovery of General Game-Playing Algorithms with Large Language Models](https://mlsanity.com/paper/2609.33115/) (2026)

## Abstract (arXiv)

We consider Monte-Carlo Tree Search (MCTS) applied to Markov Decision Processes (MDPs) and Partially Observable MDPs (POMDPs), and the well-known Upper Confidence bound for Trees (UCT) algorithm. In UCT, a tree with nodes (states) and edges (actions) is incrementally built by the expansion of nodes, and the values of nodes are updated through a backup strategy based on the average value of child nodes. However, it has been shown that with enough samples the maximum operator yields more accurate node value estimates than averaging. Instead of settling for one of these value estimates, we go a step further proposing a novel backup strategy which uses the power mean operator, which computes a value between the average and maximum value. We call our new approach Power-UCT, and argue how the use of the power mean operator helps to speed up the learning in MCTS. We theoretically analyze our method providing guarantees of convergence to the optimum. Finally, we empirically demonstrate the effectiveness of our method in well-known MDP and POMDP benchmarks, showing significant improvement in performance and convergence speed w.r.t. state of the art algorithms.