# Investigating Intra-Abstraction Policies For Non-exact Abstraction Algorithms

arXiv: https://arxiv.org/abs/2510.24297 · HTML: https://mlsanity.com/paper/2510.24297/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 29 references that are in the map.
- First posted on arXiv 28 Oct 2025.

## Builds on (most cited first, 9 of 9)

- [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019)
- [Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm](https://mlsanity.com/paper/1712.01815/) (2017)
- [Vector Quantized Models for Planning](https://mlsanity.com/paper/2106.04615/) (2021)
- [CAMPs: Learning Context-Specific Abstractions for Efficient Planning in Factored MDPs](https://mlsanity.com/paper/2007.13202/) (2020)
- [Adaptive Discretization using Voronoi Trees for Continuous POMDPs](https://mlsanity.com/paper/2302.10439/) (2023)
- [Efficient Monte Carlo Tree Search via On-the-Fly State-Conditioned Action Abstraction](https://mlsanity.com/paper/2406.00614/) (2024)
- [Time-critical and confidence-based abstraction dropping methods](https://mlsanity.com/paper/2507.02703/) (2025)
- [Investigating Scale Independent UCT Exploration Factor Strategies](https://mlsanity.com/paper/2510.21275/) (2025)
- [AUPO -- Abstracted Until Proven Otherwise: A Reward Distribution Based Abstraction Algorithm](https://mlsanity.com/paper/2510.23214/) (2025)

## Built on by (most cited first, 2 of 2)

- [Grouping Nodes With Known Value Differences: A Lossless UCT-based Abstraction Algorithm](https://mlsanity.com/paper/2510.25388/) (2025)
- [Discovering State Equivalences in UCT Search Trees By Action Pruning](https://mlsanity.com/paper/2510.26346/) (2025)

## Abstract (arXiv)

One weakness of Monte Carlo Tree Search (MCTS) is its sample efficiency which can be addressed by building and using state and/or action abstractions in parallel to the tree search such that information can be shared among nodes of the same layer. The primary usage of abstractions for MCTS is to enhance the Upper Confidence Bound (UCB) value during the tree policy by aggregating visits and returns of an abstract node. However, this direct usage of abstractions does not take the case into account where multiple actions with the same parent might be in the same abstract node, as these would then all have the same UCB value, thus requiring a tiebreak rule. In state-of-the-art abstraction algorithms such as pruned On the Go Abstractions (pruned OGA), this case has not been noticed, and a random tiebreak rule was implicitly chosen. In this paper, we propose and empirically evaluate several alternative intra-abstraction policies, several of which outperform the random policy across a majority of environments and parameter settings.