mlsanityData to 14 Oct 2026

Predictive but Not Plannable: RC-aux for Latent World Models

arXiv 2605.07278 ↗First posted 8 May 2026cs.LG cs.AI cs.CVcited by 19

On the map: World models → Latent predictive world models

Abstract (arXiv)

A latent world model may achieve accurate short-horizon prediction while still inducing a latent space that is poorly aligned with planning. A key issue is spatiotemporal mismatch: these models are often trained with local predictive supervision, but deployed for long-horizon goal-directed search in latent spaces where Euclidean distance may not reflect what is reachable within a finite action budget. We present the Reachability-Correction auxiliary objective (RC-aux), a lightweight correction for this mismatch in reconstruction-free latent world models. RC-aux keeps the world-model backbone unchanged and adds planning-aligned supervision along two axes. Along the time axis, multi-horizon open-loop prediction trains the model beyond one-step consistency. Along the space axis, budget-conditioned reachability supervision, together with temporal hard negatives, encourages the latent space to distinguish states that are eventually reachable from those reachable within the current planning horizon. At test time, the learned reachability signal can also be used by a reachability-aware planner to favor trajectories that are both goal-directed and attainable under the available budget. We instantiate RC-aux on LeWorldModel and evaluate it under both continuation-training and matched-from-scratch settings. Across goal-conditioned pixel-control tasks and a LIBERO-Goal extension, RC-aux improves LeWM-style planning with modest additional cost. These results suggest that planning with latent world models depends not only on predictive accuracy, but also on whether the learned representation encodes the temporal and geometric structure required by downstream search. The code is available at https://github.com/Guang000/RC-aux.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Placed in Latent predictive world models, one of 11 directions in World models. Builds on 42 of its 47 references that are in the map. First posted on arXiv 8 May 2026.

Builds on

42 of 47 references.

Closest ancestors · 7

Same direction, newest first

  1. 2026LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels13 Mar
  2. 2026Temporal Straightening for Latent Planning12 Mar
  3. 2025Value-guided action planning with JEPA world models28 Dec
  4. 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planning7 Nov
  5. 2024Revisiting Feature Prediction for Learning Visual Representations from Video15 Feb
  6. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture19 Jan
  7. 2018Learning Latent Dynamics for Planning from Pixels12 Nov

Other references · 35

  1. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  2. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  3. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  4. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  5. 2018World Modelscited by 1,513
  6. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
  7. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  8. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  9. 2023Mastering Diverse Domains through World Modelscited by 1,253
  10. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,057
  11. 2020Mastering Atari with Discrete World Modelscited by 1,007
  12. 2018DeepMind Control Suitecited by 956
23 more
  1. 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 883
  2. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 809
  3. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  4. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 512
  5. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 436
  6. 2020Planning to Explore via Self-Supervised World Modelscited by 368
  7. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 318
  8. 2016Deep Visual Foresight for Planning Robot Motioncited by 285
  9. 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 274
  10. 2025LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristicscited by 254
  11. 2024OGBench: Benchmarking Offline Goal-Conditioned RLcited by 238
  12. 2023Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learningcited by 121
  13. 2025Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Modelscited by 89
  14. 2021DreamerPro: Reconstruction-Free Model-Based Reinforcement Learning with Prototypical Representationscited by 74
  15. 2018Learning Plannable Representations with Causal InfoGANcited by 57
  16. 2021Temporal Predictive Coding For Model-Based Planning In Latent Spacecited by 55
  17. 2021MICo: Improved representations via sampling-based state similarity for Markov decision processescited by 43
  18. 2022DreamingV2: Reinforcement Learning with Discrete World Models without Reconstructioncited by 29
  19. 2024TLDR: Unsupervised Goal-Conditioned RL via Temporal Distance-Aware Representationscited by 17
  20. 2024MuDreamer: Learning Predictive World Models without Reconstructioncited by 13
  21. 2022State Representation Learning for Goal-Conditioned Reinforcement Learningcited by 11
  22. 2023Goal-Conditioned Reinforcement Learning with Disentanglement-based Reachability Planningcited by 5
  23. 2023PcLast: Discovering Plannable Continuous Latent Statescited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 19.

19 papers.

Sort:

  1. 2026World Model Control by Trajectory Reachability Metricscited by 8
  2. 2026Temporal-Distance JEPA: Plan-Aware Representation Learning for Latent World Model Predictive Controlcited by 8
  3. 2026Is Forward Prediction Enough? Physical State Grounding for JEPA World Modelscited by 8
  4. 2026LeFlow: Generative Latent Flow Planning for World Modelscited by 3
  5. 2026Action from Adjacent Set in Physical Space Outperforms the Best Prediction in World Modelscited by 2
  6. 2026VIScore: Diagnosing Planning-Relevant Quality in Latent World Modelscited by 2
  7. 2026Beyond Instance Slots: Semantically Rich World Models for Physical Interaction Planningcited by 2
  8. 2026Finite-Sample Metric Non-Collapse for Geometrically Supervised Latent World Models in Controlcited by 0
  9. 2026Spectral-Target Physical Latent Structuring for JEPA-Style World Modelscited by 0
  10. 2026FIRM-WM: State-factorized factual-interventional recurrent modeling for reward-free visual planningcited by 0
  11. 2026Representation World Model: Learning States, Transition and Executable Plans in Representationcited by 0
  12. 2026Adaptive Latent Capacity for World Modelscited by 0
7 more
  1. 2026Control-Geometry Straightening for Sampling-Based Latent Planningcited by 0
  2. 2026Anisotropic Representations Improve Planning in JEPA World Modelscited by 0
  3. 2026Beyond a single latent space: a dual-latent world model for long-horizon planningcited by 0
  4. 2026Supervise What Decides Success: Criterion-Aligned Auxiliary Losses for Latent World-Model Planningcited by 0
  5. 2026Keeping JEPA World Models Plannable When Little of the Frame Movescited by 0
  6. 2026How Long, Not How Close: A Learned Temporal Metric for Planning in Latent World Modelscited by 0
  7. 2026Order-Consistent Latent Geometry for Reliable Short-Horizon Planning in World Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond the Next Step: Variable-Length Latent World Models for Long-Horizon Planning
  2. 2026Latent Geometry Beyond Search: Amortizing Planning in World Models
  3. 2026ATLAS: Aligned Transport of Latent Structure for Reliable World Model Planning
  4. 2026AdaJEPA: An Adaptive Latent World Model
  5. 2026EpicWorldModel: Exploration-driven Planning with Latent World Models
  6. 2026IMWM: Intuition Models Complement World Models for Latent Planning
  7. 2020World Model as a Graph: Learning Latent Landmarks for Planning
  8. 2026SAGE: Subgoal-Conditioned Action Generation for Latent World Model Planning