AUPO -- Abstracted Until Proven Otherwise: A Reward Distribution Based Abstraction Algorithm
Not on the map. Why?
Abstract (arXiv)
We introduce a novel, drop-in modification to Monte Carlo Tree Search's (MCTS) decision policy that we call AUPO. Comparisons based on a range of IPPC benchmark problems show that AUPO clearly outperforms MCTS. AUPO is an automatic action abstraction algorithm that solely relies on reward distribution statistics acquired during the MCTS. Thus, unlike other automatic abstraction algorithms, AUPO requires neither access to transition probabilities nor does AUPO require a directed acyclic search graph to build its abstraction, allowing AUPO to detect symmetric actions that state-of-the-art frameworks like ASAP struggle with when the resulting symmetric states are far apart in state space. Furthermore, as AUPO only affects the decision policy, it is not mutually exclusive with other abstraction techniques that only affect the tree search.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 12 of its 33 references that are in the map. First posted on arXiv 27 Oct 2025.
Builds on
12 of 33 references.
References · 12
- 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
- 2014Algorithms for multi-armed bandit problemscited by 64
- 2021Vector Quantized Models for Planningcited by 48
- 2020CAMPs: Learning Context-Specific Abstractions for Efficient Planning in Factored MDPscited by 18
- 2021Decision-Making Technology for Autonomous Vehicles Learning-Based Methods, Applications and Future Outlookcited by 14
- 2022pyRDDLGym: From RDDL to Gym Environmentscited by 8
- 2023Adaptive Discretization using Voronoi Trees for Continuous POMDPscited by 6
- 2024Efficient Monte Carlo Tree Search via On-the-Fly State-Conditioned Action Abstractioncited by 6
- 2025Time-critical and confidence-based abstraction dropping methodscited by 5
- 2025Investigating Scale Independent UCT Exploration Factor Strategiescited by 4
- 2017Asymmetric Action Abstractions for Multi-Unit Control in Adversarial Real-Time Gamescited by 3
- 2023Accelerating Monte Carlo Tree Search with Probability Tree State Abstractioncited by 3
Built on by
Citing papers per year: 2025: 3.
3 papers.