# Regularizing Model-Based Planning with Energy-Based Models

arXiv: https://arxiv.org/abs/1910.05527 · HTML: https://mlsanity.com/paper/1910.05527/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Oct 2019.

## Built on by (most cited first, 11 of 11)

- [Energy-Based Imitation Learning](https://mlsanity.com/paper/2004.09395/) (2020)
- [A Survey of Embodied Learning for Object-Centric Robotic Manipulation](https://mlsanity.com/paper/2408.11537/) (2024)
- [Efficient Continuous Control with Double Actors and Regularized Critics](https://mlsanity.com/paper/2106.03050/) (2021)
- [MapGo: Model-Assisted Policy Optimization for Goal-Oriented Tasks](https://mlsanity.com/paper/2105.06350/) (2021)
- [From Screens to Scenes: A Survey of Embodied AI in Healthcare](https://mlsanity.com/paper/2501.07468/) (2025)
- [Measuring Progress in Deep Reinforcement Learning Sample Efficiency](https://mlsanity.com/paper/2102.04881/) (2021)
- [Learning from Random Demonstrations: Offline Reinforcement Learning with Importance-Sampled Diffusion Models](https://mlsanity.com/paper/2405.19878/) (2024)
- [GeoWorld: Geometric World Models](https://mlsanity.com/paper/2602.23058/) (2026)
- [Bayesian Reparameterization of Reward-Conditioned Reinforcement Learning with Energy-based Models](https://mlsanity.com/paper/2305.11340/) (2023)
- [Maximum entropy exploration in contextual bandits with neural networks and energy based models](https://mlsanity.com/paper/2210.06302/) (2022)
- [Learning to Plan from Random Exploration](https://mlsanity.com/paper/2609.38383/) (2026)

## Abstract (arXiv)

Model-based reinforcement learning could enable sample-efficient learning by quickly acquiring rich knowledge about the world and using it to improve behaviour without additional data. Learned dynamics models can be directly used for planning actions but this has been challenging because of inaccuracies in the learned models. In this paper, we focus on planning with learned dynamics models and propose to regularize it using energy estimates of state transitions in the environment. We visually demonstrate the effectiveness of the proposed method and show that off-policy training of an energy estimator can be effectively used to regularize planning with pre-trained dynamics models. Further, we demonstrate that the proposed method enables sample-efficient learning to achieve competitive performance in challenging continuous control tasks such as Half-cheetah and Ant in just a few minutes of experience.