mlsanityData to 14 Oct 2026

TransZero: Parallel Tree Expansion in MuZero using Transformer Networks

arXiv 2509.11233 ↗First posted 14 Sep 2025cs.LG cs.AIcited by 1

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

We present TransZero, a model-based reinforcement learning algorithm that removes the sequential bottleneck in Monte Carlo Tree Search (MCTS). Unlike MuZero, which constructs its search tree step by step using a recurrent dynamics model, TransZero employs a transformer-based network to generate multiple latent future states simultaneously. Combined with the Mean-Variance Constrained (MVC) evaluator that eliminates dependence on inherently sequential visitation counts, our approach enables the parallel expansion of entire subtrees during planning. Experiments in MiniGrid and LunarLander show that TransZero achieves up to an eleven-fold speedup in wall-clock time compared to MuZero while maintaining sample efficiency. These results demonstrate that parallel tree construction can substantially accelerate model-based reinforcement learning, bringing real-time decision-making in complex environments closer to practice. The code is publicly available on GitHub.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 9 of its 16 references that are in the map. First posted on arXiv 14 Sep 2025.

Builds on

9 of 16 references.

References · 9

  1. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  2. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  3. 2021Perceiver: General Perception with Iterative Attentioncited by 1,104
  4. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 734
  5. 2022Transformers are Sample-Efficient World Modelscited by 328
  6. 2021Mastering Atari Games with Limited Datacited by 237
  7. 2022TransDreamer: Reinforcement Learning with Transformer World Modelscited by 143
  8. 2024Mastering Chess with a Transformer Modelcited by 15
  9. 2024UniZero: Generalized and Efficient Planning with Scalable Latent World Modelscited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Particle Monte Carlo Tree Searchcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Variance-Aware Prior-Based Tree Policies for Monte Carlo Tree Search
  2. 2025Twice Sequential Monte Carlo for Tree Search
  3. 2024SPO: Sequential Monte Carlo Policy Optimisation
  4. 2024Highway Graph to Accelerate Reinforcement Learning
  5. 2025Policy Gradient with Tree Search: Avoiding Local Optimas through Lookahead
  6. 2024Efficient Multi-agent Reinforcement Learning by Planning
  7. 2023Forecaster: Towards Temporally Abstract Tree-Search Planning from Pixels
  8. 2026Agentic Transformers Provably Learn to Search via Reinforcement Learning