Discriminator Augmented Model-Based Reinforcement Learning
Not on the map. Why?
Abstract (arXiv)
By planning through a learned dynamics model, model-based reinforcement learning (MBRL) offers the prospect of good performance with little environment interaction. However, it is common in practice for the learned model to be inaccurate, impairing planning and leading to poor performance. This paper aims to improve planning with an importance sampling framework that accounts and corrects for discrepancy between the true and learned dynamics. This framework also motivates an alternative objective for fitting the dynamics model: to minimize the variance of value estimation during planning. We derive and implement this objective, which encourages better prediction on trajectories with larger returns. We observe empirically that our approach improves the performance of current MBRL algorithms on two stochastic control problems, and provide a theoretical basis for our method.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 21 of its 37 references that are in the map. First posted on arXiv 24 Mar 2021.
Builds on
21 of 37 references.
References · 21
- 2015Continuous control with deep reinforcement learningcited by 3,407
- 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
- 2015End-to-End Training of Deep Visuomotor Policiescited by 875
- 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
- 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
- 2019Model-Based Reinforcement Learning for Ataricited by 502
- 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 313
- 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 297
- 2019Deep Dynamics Models for Learning Dexterous Manipulationcited by 206
- 2018Sample-Efficient Reinforcement Learning with Stochastic Ensemble Value Expansioncited by 168
- 2015Learning Continuous Control Policies by Stochastic Value Gradientscited by 162
- 2019Benchmarking Model-Based Reinforcement Learningcited by 162
9 more
- 2018Model-Based Value Estimation for Efficient Model-Free Reinforcement Learningcited by 156
- 2015Model Predictive Path Integral Control using Covariance Variable Importance Samplingcited by 124
- 2018Plan Online, Learn Offline: Efficient Learning and Exploration via Model-Based Controlcited by 113
- 2015Gaussian Processes for Data-Efficient Learning in Robotics and Controlcited by 98
- 2020Off-Dynamics Reinforcement Learning: Training for Transfer with Domain Classifierscited by 97
- 2020Objective Mismatch in Model-based Reinforcement Learningcited by 96
- 2019Exploring Model-based Planning with Policy Networkscited by 90
- 2019Bias Correction of Learned Generative Models using Likelihood-Free Importance Weightingcited by 70
- 2020Goal-Aware Prediction: Learning to Model What Matterscited by 45
Built on by
Citing papers per year: 2022: 1; 2023: 1.
2 papers.