mlsanityData to 14 Oct 2026

Adaptive Mixing of Policies from Searching and Policies from Learning

arXiv 2608.15700 ↗First posted 16 Aug 2026cs.AIcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Background: Distillation of training targets generated thru search/planning has proven useful in reinforcement learning, but search can take exceedingly long. Objectives: Rather than perform search to the same depth every time (typically at a fixed period of steps), reduce the search depth proportionally to the quality of the policy network priors. Methods: We describe Flexer, an architecture that, for each step, mixes the policy from a neural network and the policy from Monte Carlo tree search. The mixing factor favors the MCTS policy as the policy imitation error of the network and the environment models' variance increases. Results: Flexer outperforms a version of AlphaZero (and DQN and ADP) for some experiments on three toy symbolic problems.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 11 of its 21 references that are in the map. First posted on arXiv 16 Aug 2026.

Builds on

11 of 21 references.

References · 11

  1. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  2. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  3. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  4. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
  5. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
  6. 2024ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Searchcited by 344
  7. 2017Thinking Fast and Slow with Deep Learning and Tree Searchcited by 252
  8. 2018Deep Variational Reinforcement Learning for POMDPscited by 125
  9. 2018How to Combine Tree-Search Methods in Reinforcement Learningcited by 16
  10. 2023BetaZero: Belief-State Planning for Long-Horizon POMDPs using Learned Approximationscited by 14
  11. 2022Policy Gradient Algorithms with Monte Carlo Tree Learning for Non-Markov Decision Processescited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Policy Gradient with Tree Search: Avoiding Local Optimas through Lookahead
  2. 2026Hindsight-Anchored Policy Optimization: Learning Through Hindsight with Thompson Sampling-Inspired Adaptive Gating
  3. 2026StarOR: Synergizing Tree Search and Test-Time Reinforcement Learning for Optimization Modeling
  4. 2024Decision Making in Non-Stationary Environments with Policy-Augmented Search
  5. 2024BlendRL: A Framework for Merging Symbolic and Neural Policy Learning
  6. 2025Squeeze the Soaked Sponge: Efficient Off-policy Reinforcement Finetuning for Large Language Model
  7. 2025Stabilizing Reinforcement Learning with LLMs: Formulation and Practices
  8. 2024Enhancing Reinforcement Learning Through Guided Search