mlsanityData to 14 Oct 2026

Latent Energy Action Planning with World Models

arXiv 2609.03294 ↗First posted 3 Sep 2026cs.LGcited by 4

On the map: World models → Latent predictive world models

Abstract (arXiv)

Latent world models support efficient model predictive control from high-dimensional observations, yet optimizing a single learned latent objective can favor action sequences whose decoder-predicted terminal descriptor does not match the goal descriptor. We introduce Latent Energy Action Planning (LEAP), which treats the complete action horizon as a differentiable variable and optimizes it through a frozen LeWorldModel (LeWM). LEAP couples terminal latent goal matching with a terminal-window state energy. Low energy requires the predicted terminal latent to agree with the goal latent and the decoder-predicted terminal descriptor to agree with the goal descriptor. A frozen goal-conditioned proposal initializes the search, a quasi-Newton solver refines actions through the autoregressive rollout, and post-optimization projection enforces the admissible action range. Across four control domains using the officially released LeWM checkpoints, the complete LEAP planning system raises mean success from 77.5% for LeWM planned with the cross-entropy method (LeWM+CEM) to 94.8% under a matched protocol, a 17.3-percentage-point improvement, while retaining the frozen LeWM representation and predictor.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Latent predictive world models, one of 11 directions in World models. Builds on 17 of its 22 references that are in the map. First posted on arXiv 3 Sep 2026.

Builds on

17 of 22 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels13 Mar
  2. 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planning7 Nov
  3. 2024Revisiting Feature Prediction for Learning Visual Representations from Video15 Feb
  4. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture19 Jan
  5. 2022Temporal Difference Learning for Model Predictive Control9 Mar
  6. 2018Learning Latent Dynamics for Planning from Pixels12 Nov

Other references · 11

  1. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  2. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  3. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 840
  4. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
  5. 2020MOPO: Model-based Offline Policy Optimizationcited by 636
  6. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 590
  7. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 512
  8. 2021Implicit Behavioral Cloningcited by 452
  9. 2018Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Controlcited by 230
  10. 2018Differentiable MPC for End-to-end Planning and Controlcited by 189
  11. 2021Composable Energy Policies for Reactive Motion Generation and Reinforcement Learningcited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Beyond a single latent space: a dual-latent world model for long-horizon planningcited by 0
  2. 2026Learning to Plan from Random Explorationcited by 0
  3. 2026When Low Prediction Error Misleads Planning: Diagnosing Representation, Dynamics, and Decision Failures in Latent World Modelscited by 0
  4. 2026Parallel Predictive World Models for Accurate and Efficient Long-Horizon Planningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026SAGE: Subgoal-Conditioned Action Generation for Latent World Model Planning
  2. 2026Latent Geometry Beyond Search: Amortizing Planning in World Models
  3. 2025Planning as Descent: Goal-Conditioned Latent Trajectory Synthesis in Learned Energy Landscapes
  4. 2026LaWM: Least Action World Models for Long-Horizon Physical Consistency from Visual Observations
  5. 2026Decision-Metric Alignment in Latent World Models: Diagnostics and Action-Conditioned Objectives for MPC Planning
  6. 2026Energy-Structured Latent World Models with Neural Time Fields for Physically Constistent Open-World Motion Planning
  7. 2026Beyond the Next Step: Variable-Length Latent World Models for Long-Horizon Planning
  8. 2026AD-WM: Action-Discriminative World Models for Counterfactual Model Predictive Control