mlsanityData to 7 Oct 2026

EfficientZero V2: Mastering Discrete and Continuous Control with Limited Data

arXiv 2403.00564 ↗First posted 1 Mar 2024cs.LG cs.AI cs.ROcited by 39

On the map: World models → Latent dynamics models and model-based RL

Abstract (arXiv)

Sample efficiency remains a crucial challenge in applying Reinforcement Learning (RL) to real-world tasks. While recent algorithms have made significant strides in improving sample efficiency, none have achieved consistently superior performance across diverse domains. In this paper, we introduce EfficientZero V2, a general framework designed for sample-efficient RL algorithms. We have expanded the performance of EfficientZero to multiple domains, encompassing both continuous and discrete actions, as well as visual and low-dimensional inputs. With a series of improvements we propose, EfficientZero V2 outperforms the current state-of-the-art (SOTA) by a significant margin in diverse tasks under the limited data setting. EfficientZero V2 exhibits a notable advancement over the prevailing general algorithm, DreamerV3, achieving superior outcomes in 50 of 66 evaluated tasks across diverse benchmarks, such as Atari 100k, Proprio Control, and Vision Control.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 28 of its 38 references that are in the map. First posted on arXiv 1 Mar 2024.

Builds on

28 of 38 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023Mastering Diverse Domains through World Models10 Jan
  2. 2021Mastering Atari Games with Limited Data30 Oct
  3. 2020Mastering Atari with Discrete World Models5 Oct
  4. 2019Dream to Control: Learning Behaviors by Latent Imagination3 Dec

Other references · 24

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2015Continuous control with deep reinforcement learningcited by 3,279
  3. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  4. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  5. 2018DeepMind Control Suitecited by 935
  6. 2020On Layer Normalization in the Transformer Architecturecited by 906
  7. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  8. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 723
  9. 2019Solving Rubik's Cube with a Robot Handcited by 699
  10. 2018Learning Dexterous In-Hand Manipulationcited by 643
  11. 2019Model-Based Reinforcement Learning for Ataricited by 495
  12. 2019Learning agile and dynamic motor skills for legged robotscited by 402
12 more
  1. 2022Temporal Difference Learning for Model Predictive Controlcited by 375
  2. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
  3. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 309
  4. 2019Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacementcited by 205
  5. 2021A System for General In-Hand Object Re-Orientationcited by 155
  6. 2018Model-Based Value Estimation for Efficient Model-Free Reinforcement Learningcited by 152
  7. 2023Bigger, Better, Faster: Human-level Atari with human-level efficiencycited by 145
  8. 2022Visual Dexterity: In-Hand Reorientation of Novel and Complex Object Shapescited by 92
  9. 2021Online and Offline Reinforcement Learning by Planning with a Learned Modelcited by 90
  10. 2021Learning and Planning in Complex Action Spacescited by 76
  11. 2017Multi-step Reinforcement Learning: A Unifying Algorithmcited by 42
  12. 2023DexPBT: Scaling up Dexterous Manipulation for Hand-Arm Systems with Population Based Trainingcited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 21; 2026: 15.

39 papers.

Sort:

By area:

  1. 2024Bigger, Regularized, Optimistic: scaling for compute and sample-efficient continuous controlcited by 104
  2. 2025Towards General-Purpose Model-Free Reinforcement Learningcited by 40
  3. 2025Video2Policy: Scaling up Manipulation Tasks in Simulation through Internet Videoscited by 21
  4. 2025Learning Transformer-based World Models with Contrastive Predictive Codingcited by 20
  5. 2025A Step Toward World Models: A Survey on Robotic Manipulationcited by 15
  6. 2024Scaling Offline Model-Based RL via Jointly-Optimized World-Action Model Pretrainingcited by 13
  7. 2025Bootstrapped Model Predictive Controlcited by 13
  8. 2026R2-Dreamer: Redundancy-Reduced World Models without Decoders or Augmentationcited by 10
  9. 2025Multi-Task Reinforcement Learning Enables Parameter Scalingcited by 8
  10. 2024Beyond The Rainbow: High Performance Deep Reinforcement Learning on a Desktop PCcited by 7
  11. 2025Accurate and Efficient World Modeling with Masked Latent Transformerscited by 7
  12. 2025Simulus: Combining Improvements in Sample-Efficient World Model Agentscited by 6
27 more
  1. 2025Hadamax Encoding: Elevating Performance in Model-Free Ataricited by 6
  2. 2025Trust-Region Twisted Policy Improvementcited by 5
  3. 2025TimeRewarder: Learning Dense Reward from Passive Videos via Frame-wise Temporal Distancecited by 5
  4. 2026The Surprising Difficulty of Search in Model-Based Reinforcement Learningcited by 5
  5. 2025AXIOM: Learning to Play Games in Minutes with Expanding Object-Centric Modelscited by 3
  6. 2025Reliability-Adjusted Prioritized Experience Replaycited by 3
  7. 2026World Models: A Comprehensive Survey of Architectures, Methodologies, Reasoning Paradigms, and Applicationscited by 3
  8. 2025A KL-regularization Framework for Learning to Plan with Adaptive Priorscited by 2
  9. 2025Human-Like Goalkeeping in a Realistic Football Simulation: a Sample-Efficient Reinforcement Learning Approachcited by 2
  10. 2025Planning in Branch-and-Bound: Model-Based Reinforcement Learning for Exact Combinatorial Optimizationcited by 2
  11. 2026Optimistic World Models: Efficient Exploration in Model-Based Deep Reinforcement Learningcited by 2
  12. 2026Dreamer-CDP: Improving Reconstruction-free World Models Via Continuous Deterministic Representation Predictioncited by 2
  13. 2026EfficientTDMPC: Improved MPC Objectives for Sample-Efficient Continuous Controlcited by 2
  14. 2025Sample-Efficient Reinforcement Learning Controller for Deep Brain Stimulation in Parkinson's Diseasecited by 1
  15. 2026Object-Centric World Models Meet Monte Carlo Tree Searchcited by 1
  16. 2026What Capable Agents Must Know: Selection Theorems for Robust Decision-Making under Uncertaintycited by 1
  17. 2026Reason to Play: Behavioral and Brain Alignment Between Frontier LRMs and Human Game Learnerscited by 1
  18. 2026PriorZero: Bridging Language Priors and World Models for Decision Makingcited by 1
  19. 2025Coupled Distributional Random Expert Distillation for World Model Online Imitation Learningcited by 0
  20. 2025Predictive Spike Timing Enables Distributed Shortest Path Computation in Spiking Neural Networkscited by 0
  21. 2025Guardian: Decoupling Exploration from Safety in Reinforcement Learningcited by 0
  22. 2025Recurrent Off-Policy Deep Reinforcement Learning Doesn't Have to be Slowcited by 0
  23. 2026Scaling Tasks, Not Samples: Mastering Humanoid Control through Multi-Task Model-Based Reinforcement Learningcited by 0
  24. 2026Particle Monte Carlo Tree Searchcited by 0
  25. 2026ProDVI: Programmatic Dynamics Priors for Value Network Initializationcited by 0
  26. 2026CausalNav: Reliability-Certified Causal World Models for Control under Physical-Parameter Shiftcited by 0
  27. 2026Towards Optimal Policy Improvementcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024On Sample-Efficient Offline Reinforcement Learning: Data Diversity, Posterior Sampling, and Beyond
  2. 2024Towards Sample-Efficiency and Generalization of Transfer and Inverse Reinforcement Learning: A Comprehensive Literature Review
  3. 2026DataFlex-RL: An Evaluation Platform for RLVR Data Policies
  4. 2026FASTER: Value-Guided Sampling for Fast RL
  5. 2025Statistical and Algorithmic Foundations of Reinforcement Learning
  6. 2022MoDem: Accelerating Visual Model-Based Reinforcement Learning with Demonstrations
  7. 2024Enhancing Sample Efficiency and Exploration in Reinforcement Learning through the Integration of Diffusion Models and Proximal Policy Optimization
  8. 2022On Practical Reinforcement Learning: Provable Robustness, Scalability, and Statistical Efficiency