# Sample-efficient Cross-Entropy Method for Real-time Planning

arXiv: https://arxiv.org/abs/2008.06389 · HTML: https://mlsanity.com/paper/2008.06389/

## Key facts

- Cited by 95 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Aug 2020.

## Built on by (most cited first, 30 of 95)

- [Optimistic Active Exploration of Dynamical Systems](https://mlsanity.com/paper/2306.12371/) (2023)
- [Curious Exploration via Structured World Models Yields Zero-Shot Object Manipulation](https://mlsanity.com/paper/2206.11403/) (2022)
- [DEP-RL: Embodied Exploration for Reinforcement Learning in Overactuated and Musculoskeletal Systems](https://mlsanity.com/paper/2206.00484/) (2022)
- [Learning Getting-Up Policies for Real-World Humanoid Robots](https://mlsanity.com/paper/2502.12152/) (2025)
- [Combining Evolution and Deep Reinforcement Learning for Policy Search: a Survey](https://mlsanity.com/paper/2203.14009/) (2022)
- [ActSafe: Active Exploration with Safety Constraints for Reinforcement Learning](https://mlsanity.com/paper/2410.09486/) (2024)
- [Learning Dynamics Models for Model Predictive Agents](https://mlsanity.com/paper/2109.14311/) (2021)
- [Action Noise in Off-Policy Deep Reinforcement Learning: Impact on Exploration and Performance](https://mlsanity.com/paper/2206.03787/) (2022)
- [Reconstruction or Semantics? What Makes a Latent Space Useful for Robotic World Models](https://mlsanity.com/paper/2605.06388/) (2026)
- [stable-worldmodel: A Platform for Reproducible World Modeling Research and Evaluation](https://mlsanity.com/paper/2605.21800/) (2026)
- [Sparsely Changing Latent States for Prediction and Planning in Partially Observable Domains](https://mlsanity.com/paper/2110.15949/) (2021)
- [Style-ERD: Responsive and Coherent Online Motion Style Transfer](https://mlsanity.com/paper/2203.02574/) (2022)
- [Distributionally Robust Model-based Reinforcement Learning with Large State Spaces](https://mlsanity.com/paper/2309.02236/) (2023)
- [An Experimental Design Perspective on Model-Based Reinforcement Learning](https://mlsanity.com/paper/2112.05244/) (2021)
- [Latent Geometry Beyond Search: Amortizing Planning in World Models](https://mlsanity.com/paper/2605.08732/) (2026)
- [Variational Inference MPC using Normalizing Flows and Out-of-Distribution Projection](https://mlsanity.com/paper/2205.04667/) (2022)
- [Learning to Get Up](https://mlsanity.com/paper/2205.00307/) (2022)
- [Inferring Smooth Control: Monte Carlo Posterior Policy Iteration with Gaussian Processes](https://mlsanity.com/paper/2210.03512/) (2022)
- [Learning Sampling Distributions for Model Predictive Control](https://mlsanity.com/paper/2212.02587/) (2022)
- [CEM-GD: Cross-Entropy Method with Gradient Descent Planner for Model-Based Reinforcement Learning](https://mlsanity.com/paper/2112.07746/) (2021)
- [Exploration via Planning for Information about the Optimal Trajectory](https://mlsanity.com/paper/2210.04642/) (2022)
- [Hierarchical Policy Blending as Inference for Reactive Robot Control](https://mlsanity.com/paper/2210.07890/) (2022)
- [Constrained Stein Variational Trajectory Optimization](https://mlsanity.com/paper/2308.12110/) (2023)
- [Continual Reinforcement Learning by Planning with Online World Models](https://mlsanity.com/paper/2507.09177/) (2025)
- [Test-time Offline Reinforcement Learning on Goal-related Experience](https://mlsanity.com/paper/2507.18809/) (2025)
- [Temporal-Distance JEPA: Plan-Aware Representation Learning for Latent World Model Predictive Control](https://mlsanity.com/paper/2607.25337/) (2026)
- [A Pontryagin Perspective on Reinforcement Learning](https://mlsanity.com/paper/2405.18100/) (2024)
- [SOMBRL: Scalable and Optimistic Model-Based RL](https://mlsanity.com/paper/2511.20066/) (2025)
- [Parallel Stochastic Gradient-Based Planning for World Models](https://mlsanity.com/paper/2602.00475/) (2026)
- [Test-Time Trajectory Optimization for Autonomous Driving](https://mlsanity.com/paper/2606.07170/) (2026)

## Abstract (arXiv)

Trajectory optimizers for model-based reinforcement learning, such as the Cross-Entropy Method (CEM), can yield compelling results even in high-dimensional control tasks and sparse-reward environments. However, their sampling inefficiency prevents them from being used for real-time planning and control. We propose an improved version of the CEM algorithm for fast planning, with novel additions including temporally-correlated actions and memory, requiring 2.7-22x less samples and yielding a performance increase of 1.2-10x in high-dimensional control problems.