mlsanityData to 7 Oct 2026

Variational Inference MPC for Bayesian Model-based Reinforcement Learning

arXiv 1907.04202 ↗First posted 8 Jul 2019cs.LG cs.SY eess.SYcited by 39

Not on the map. Why?

Abstract (arXiv)

In recent studies on model-based reinforcement learning (MBRL), incorporating uncertainty in forward dynamics is a state-of-the-art strategy to enhance learning performance, making MBRLs competitive to cutting-edge model free methods, especially in simulated robotics tasks. Probabilistic ensembles with trajectory sampling (PETS) is a leading type of MBRL, which employs Bayesian inference to dynamics modeling and model predictive control (MPC) with stochastic optimization via the cross entropy method (CEM). In this paper, we propose a novel extension to the uncertainty-aware MBRL. Our main contributions are twofold: Firstly, we introduce a variational inference MPC, which reformulates various stochastic methods, including CEM, in a Bayesian fashion. Secondly, we propose a novel instance of the framework, called probabilistic action ensembles with trajectory sampling (PaETS). As a result, our Bayesian MBRL can involve multimodal uncertainties both in dynamics and optimal trajectories. In comparison to PETS, our method consistently improves asymptotic performance on several challenging locomotion tasks.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Jul 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 10; 2022: 6; 2023: 8; 2024: 4; 2025: 9; 2026: 1.

39 papers.

Sort:

By area:

  1. 2021Active Inference in Robotics and Artificial Agents: Survey and Challengescited by 52
  2. 2023Generalized Schr"odinger Bridge Matchingcited by 40
  3. 2022DreamingV2: Reinforcement Learning with Discrete World Models without Reconstructioncited by 29
  4. 2023Efficient Preference-Based Reinforcement Learning Using Learned Dynamics Modelscited by 20
  5. 2021Co-Adaptation of Algorithmic and Implementational Innovations in Inference-based Deep Reinforcement Learningcited by 15
  6. 2022Variational Inference MPC using Normalizing Flows and Out-of-Distribution Projectioncited by 13
  7. 2021Variational Inference MPC using Tsallis Divergencecited by 11
  8. 2021Understanding the Origin of Information-Seeking Exploration in Probabilistic Objectives for Controlcited by 10
  9. 2021Dual Online Stein Variational Inference for Control and Dynamicscited by 10
  10. 2022Inferring Smooth Control: Monte Carlo Posterior Policy Iteration with Gaussian Processescited by 10
  11. 2022Learning Sampling Distributions for Model Predictive Controlcited by 10
  12. 2023Efficient Dynamics Modeling in Interactive Environments with Koopman Theorycited by 10
27 more
  1. 2021Applications of the Free Energy Principle to Machine Learning and Neurosciencecited by 9
  2. 2023Mind the Model, Not the Agent: The Primacy Bias in Model-based RLcited by 9
  3. 2023Constrained Stein Variational Trajectory Optimizationcited by 8
  4. 2020Domain-Adversarial and Conditional State Space Model for Imitation Learningcited by 7
  5. 2023Deep Model Predictive Optimizationcited by 7
  6. 2024Model-based Reinforcement Learning for Parameterized Action Spacescited by 7
  7. 2021Optimistic Reinforcement Learning by Forward Kullback-Leibler Divergence Optimizationcited by 6
  8. 2021CEM-GD: Cross-Entropy Method with Gradient Descent Planner for Model-Based Reinforcement Learningcited by 6
  9. 2025Guaranteeing Out-Of-Distribution Detection in Deep RL via Transition Estimationcited by 6
  10. 2022A Simple Decentralized Cross-Entropy Methodcited by 5
  11. 2021A Whole Brain Probabilistic Generative Model: Toward Realizing Cognitive Architectures for Developmental Robotscited by 4
  12. 2025Reflect-then-Plan: Offline Model-Based Planning through a Doubly Bayesian Lenscited by 4
  13. 2024Risk-sensitive control as inference with R'enyi divergencecited by 3
  14. 2025AXIOM: Learning to Play Games in Minutes with Expanding Object-Centric Modelscited by 3
  15. 2024A Contact Model based on Denoising Diffusion to Learn Variable Impedance Control for Contact-rich Manipulationcited by 2
  16. 2023Probabilistic Constrained Reinforcement Learning with Formal Interpretabilitycited by 1
  17. 2024Design of Restricted Normalizing Flow towards Arbitrary Stochastic Policy with Computational Efficiencycited by 1
  18. 2025DR-PETS: Learning-Based Control With Planning in Adversarial Environmentscited by 1
  19. 2025Improving the Resilience of Quadrotors in Underground Environments by Combining Learning-based and Safety Controllerscited by 1
  20. 2026ELVIS: Ensemble-Calibrated Latent Imagination for Long-Horizon Visual MPCcited by 1
  21. 2021ED2: Environment Dynamics Decomposition World Models for Continuous Controlcited by 0
  22. 2022Sparse Representation Learning with Modified q-VAE towards Minimal Realization of World Modelcited by 0
  23. 2023KEEC: Koopman Embedded Equivariant Controlcited by 0
  24. 2025Safe Beyond the Horizon: Efficient Sampling-based MPC with Neural Control Barrier Functionscited by 0
  25. 2025Bregman Centroid Guided Cross-Entropy Methodcited by 0
  26. 2025Q-Guided Stein Variational Model Predictive Control via RL-informed Policy Priorcited by 0
  27. 2025Learning to Plan, Planning to Learn: Adaptive Hierarchical RL-MPC for Sample-Efficient Decision Makingcited by 0