mlsanityData to 7 Oct 2026

A0C: Alpha Zero in Continuous Action Space

arXiv 1805.09613 ↗First posted 24 May 2018stat.ML cs.AI cs.LGcited by 17

On the map: World models → Latent dynamics models and model-based RL

Abstract (arXiv)

A core novelty of Alpha Zero is the interleaving of tree search and deep learning, which has proven very successful in board games like Chess, Shogi and Go. These games have a discrete action space. However, many real-world reinforcement learning domains have continuous action spaces, for example in robotic control, navigation and self-driving cars. This paper presents the necessary theoretical extensions of Alpha Zero to deal with continuous action space. We also provide some preliminary experiments on the Pendulum swing-up task, empirically showing the feasibility of our approach. Thereby, this work provides a first step towards the application of iterated search and learning in domains with a continuous action space.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 0 of its 22 references that are in the map. First posted on arXiv 24 May 2018.

Builds on

0 of 22 references.

No references here yet.

Built on by

Cited by year

Citing papers per year: 2021: 5; 2022: 5; 2023: 2; 2024: 5.

17 papers.

Sort:

By area:

  1. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 161
  2. 2021Learning and Planning in Complex Action Spacescited by 76
  3. 2022A Survey on Model-based Reinforcement Learningcited by 65
  4. 2023Beyond Games: A Systematic Review of Neural Monte Carlo Tree Search Applicationscited by 16
  5. 2021High-Accuracy Model-Based Reinforcement Learning, a Surveycited by 13
  6. 2023BetaZero: Belief-State Planning for Long-Horizon POMDPs using Learned Approximationscited by 13
  7. 2021Lyapunov Exponents for Diversity in Differentiable Gamescited by 8
  8. 2024ConstrainedZero: Chance-Constrained POMDP Planning using Learned Probabilistic Failure Surrogates and Adaptive Safety Constraintscited by 7
  9. 2021AlphaDDA: Strategies for Adjusting the Playing Strength of a Fully Trained AlphaZero System to a Suitable Human Training Partnercited by 5
  10. 2022Learning to design without prior data: Discovering generalizable design strategies using deep learning and tree searchcited by 5
  11. 2021Combining Off and On-Policy Training in Model-Based Reinforcement Learningcited by 4
  12. 2022Continuous Monte Carlo Graph Searchcited by 4
5 more
  1. 2024SPO: Sequential Monte Carlo Policy Optimisationcited by 4
  2. 2022GrASP: Gradient-Based Affordance Selection for Planningcited by 2
  3. 2022Brick Tic-Tac-Toe: Exploring the Generalizability of AlphaZero to Novel Test Environmentscited by 2
  4. 2024Provably Efficient Long-Horizon Exploration in Monte Carlo Tree Search through State Occupancy Regularizationcited by 1
  5. 2024AlphaViT: A flexible game-playing AI for multiple games and variable board sizescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024An Investigation of Offline Reinforcement Learning in Factorisable Action Spaces
  2. 2025Scaling Algorithm Distillation for Continuous Control with Mamba
  3. 2026Synthesizing Reactive Character Behaviors for Continuous Games via Programmatic Policy Search
  4. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
  5. 2021Continuous Control with Action Quantization from Demonstrations
  6. 2026Improving and Accelerating Offline RL in Large Discrete Action Spaces with Structured Policy Initialization
  7. 2025Implicit Search via Discrete Diffusion: A Study on Chess
  8. 2025Action Space Reduction Strategies for Reinforcement Learning in Autonomous Driving