Two-Fidelity Best-Action Identification for Stochastic Minimax Tree
Not on the map. Why?
Abstract (arXiv)
We study fixed-confidence best-action identification (BAI) in stochastic minimax trees. This problem is increasingly relevant in modern AI planning, where deep minimax search and Monte Carlo Tree Search (MCTS) with language model long rollouts face a fundamental tradeoff: heuristic evaluations are cheap but biased, while accurate rollouts are reliable but prohibitively expensive. We propose 2FFS, a two-fidelity tree-search algorithm that brings multi-fidelity flat bandit ideas into trees. The algorithm combines minimax-style fast expansion with MCTS-style stochastic sampling, adaptively deciding when to exploit cheap biased evaluations and when to invoke expensive accurate evaluations for local certification. We prove fixed-confidence correctness, establish finite stopping for exact identification, and give a polynomial-depth cost upper bound for general-depth trees. Across numerical stochastic-tree experiments, 2FFS uses substantially fewer samples and computational operations comparing to existing BAI-MCTS baseline.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 30 of its 47 references that are in the map. First posted on arXiv 1 Jun 2026.
Builds on
30 of 47 references.
References · 30
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
- 2016Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimizationcited by 688
- 2018BOHB: Robust and Efficient Hyperparameter Optimization at Scalecited by 466
- 2014On the Complexity of Best Arm Identification in Multi-Armed Bandit Modelscited by 280
- 2016Optimal Best Arm Identification with Fixed Confidencecited by 233
- 2018Time-uniform, nonparametric, nonasymptotic confidence sequencescited by 220
- 2015Non-stochastic Best Arm Identification and Hyperparameter Optimizationcited by 219
- 2018A System for Massively Parallel Hyperparameter Tuningcited by 201
- 2013lil' UCB : An Optimal Exploration Algorithm for Multi-Armed Banditscited by 148
- 2017Multi-fidelity Bayesian Optimisation with Continuous Approximationscited by 120
- 2016Multi-Information Source Optimizationcited by 83
- 2018Mixture Martingales Revisited with Applications to Sequential Tests and Confidence Intervalscited by 76
18 more
- 2019Non-Asymptotic Pure Exploration by Solving Gamescited by 62
- 2020Multi-Fidelity Bayesian Optimization via Deep Neural Networkscited by 46
- 2023Zeroth-Order Optimization Meets Human Feedback: Provable Learning via Ranking Oraclescited by 34
- 2025Exploration vs Exploitation: Rethinking RLVR through Clipping, Entropy, and Spurious Rewardcited by 25
- 2016The Multi-fidelity Multi-armed Banditcited by 22
- 2025Stepwise Guided Policy Optimization: Coloring your Incorrect Reasoning in GRPOcited by 18
- 2024Two-Timescale Gradient Descent Ascent Algorithms for Nonconvex Minimax Optimizationcited by 15
- 2017Monte-Carlo Tree Search by Best Arm Identificationcited by 13
- 2024Cost Aware Best Arm Identificationcited by 12
- 2014Monte Carlo Tree Search with Heuristic Evaluations using Implicit Minimax Backupscited by 11
- 2025ComPO: Preference Alignment via Comparison Oraclescited by 11
- 2026Reward-free Alignment for Conflicting Objectivescited by 10
- 2018Noisy Blackbox Optimization with Multi-Fidelity Queries: A Tree Search Approachcited by 9
- 2024Optimal Multi-Fidelity Best-Arm Identificationcited by 8
- 2023Multi-Fidelity Multi-Armed Bandits Revisitedcited by 6
- 2024Best-Arm Identification in Unimodal Banditscited by 6
- 2025Post-Training LLMs as Better Decision-Making Agents: A Regret-Minimization Approachcited by 4
- 2026Adaptive multi-fidelity optimization with fast learning ratescited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
- 2026A Zeroth-Order Paradigm for LLM Preference Alignmentcited by 0
- 2026A Zeroth-Order Paradigm for LLM Preference Alignment16 Sep 2026