Data to 7 Oct 2026

Sample-Efficient Reinforcement Learning with Stochastic Ensemble Value Expansion

arXiv 1807.01675 ↗First posted 4 Jul 2018cs.LG cs.AI stat.MLcited by 167

Not on the map. Why?

Abstract (arXiv)

Integrating model-free and model-based approaches in reinforcement learning has the potential to achieve the high performance of model-free algorithms with low sample complexity. However, this is difficult because an imperfect dynamics model can degrade the performance of the learning algorithm, and in sufficiently complex environments, the dynamics model will almost always be imperfect. As a result, a key challenge is to combine model-based approaches with model-free learning in such a way that errors in the model do not degrade performance. We propose stochastic ensemble value expansion (STEVE), a novel model-based technique that addresses this issue. By dynamically interpolating between model rollouts of various horizon lengths for each individual example, STEVE ensures that the model is only utilized when doing so does not introduce significant errors. Our approach outperforms model-free baselines on challenging continuous control benchmarks with an order-of-magnitude increase in sample efficiency, and in contrast to previous model-based approaches, performance does not degrade in complex environments.

Key facts. Cited by 167 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Jul 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 44; 2022: 24; 2023: 44; 2024: 30; 2025: 11; 2026: 14.

167 papers.

Sort:

By area:

2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
2022Temporal Difference Learning for Model Predictive Controlcited by 375
2021Randomized Ensembled Double Q-Learning: Learning Fast Without a Modelcited by 278
2021Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skillscited by 125
2021Learning to drive from a world on railscited by 114
2021Generalized Decision Transformer for Offline Hindsight Information Matchingcited by 105
2022Accelerated Policy Learning with Parallel Differentiable Simulationcited by 92
2021Gradients are Not All You Needcited by 82
2024Simple Hierarchical Planning with Diffusioncited by 71
2022A Survey on Model-based Reinforcement Learningcited by 65
2022VRL3: A Data-Driven Framework for Visual Deep Reinforcement Learningcited by 51
2021Revisiting Design Choices in Offline Model-Based Reinforcement Learningcited by 50

38 more

2023Sample-Efficient Multi-Objective Learning via Generalized Policy Improvement Prioritizationcited by 46
2021Visual Adversarial Imitation Learning using Variational Modelscited by 42
2023Multi-agent Reinforcement Learning: A Comprehensive Surveycited by 30
2025Uncertainty-aware Latent Safety Filters for Avoiding Out-of-Distribution Failurescited by 30
2023Rethinking Closed-loop Training for Autonomous Drivingcited by 29
2022Simplifying Model-based RL: Learning Representations, Latent-space Models, and Policies with One Objectivecited by 28
2022Disentangling Epistemic and Aleatoric Uncertainty in Reinforcement Learningcited by 27
2021Principled Exploration via Optimistic Bootstrapping and Backward Inductioncited by 26
2022Controlling Commercial Cooling Systems Using Reinforcement Learningcited by 25
2024ODRL: A Benchmark for Off-Dynamics Reinforcement Learningcited by 25
2021Model-Based Opponent Modelingcited by 21
2021Learning Dynamics Models for Model Predictive Agentscited by 21
2022Double Check Your State Before Trusting It: Confidence-Aware Bidirectional Offline Model-Based Imaginationcited by 21
2021When should agents explore?cited by 20
2023Seizing Serendipity: Exploiting the Value of Past Success in Off-Policy Actor-Criticcited by 20
2023LightZero: A Unified Benchmark for Monte Carlo Tree Search in General Sequential Decision Scenarioscited by 20
2023Is Model Ensemble Necessary? Model-based RL via a Single Model with Lipschitz Regularized Value Functioncited by 19
2021A Survey on Deep Reinforcement Learning for Audio-Based Applicationscited by 17
2021Value Iteration in Continuous Actions, States and Timecited by 17
2021Sample Efficient Reinforcement Learning via Model-Ensemble Exploration and Exploitationcited by 17
2022The Benefits of Model-Based Generalization in Reinforcement Learningcited by 17
2024Trust the Model Where It Trusts Itself -- Model-Based Actor-Critic with Uncertainty-Aware Rollout Adaptioncited by 17
2022Conservative Bayesian Model-Based Value Expansion for Offline Policy Optimizationcited by 16
2023Plan To Predict: Learning an Uncertainty-Foreseeing Model for Model-Based Reinforcement Learningcited by 16
2023Ensemble Reinforcement Learning: A Surveycited by 16
2024MAD-TD: Model-Augmented Data stabilizes High Update Ratio RLcited by 16
2021Autoregressive Quantile Flows for Predictive Uncertainty Estimationcited by 15
2022Model-based Trajectory Stitching for Improved Offline Reinforcement Learningcited by 15
2023Model-Based Uncertainty in Value Functionscited by 15
2021MHER: Model-based Hindsight Experience Replaycited by 14
2022When to Update Your Model: Constrained Model-based Reinforcement Learningcited by 14
2023COPlanner: Plan to Roll Out Conservatively but to Explore Optimistically for Model-Based RLcited by 14
2021MapGo: Model-Assisted Policy Optimization for Goal-Oriented Taskscited by 13
2021On Effective Scheduling of Model-based Reinforcement Learningcited by 13
2024MOTO: Offline Pre-training to Online Fine-tuning for Model-based Robot Learningcited by 13
2023Off-Policy RL Algorithms Can be Sample-Efficient for Continuous Control via Sample Multiple Reusecited by 12
2023Censored Sampling of Diffusion Models Using 3 Minutes of Human Feedbackcited by 12
2021Centralized Model and Exploration Policy for Multi-Agent RLcited by 11

50 most cited of 167 All 167 (JSON)