The Intervention Gap in Latent World Models
On the map: World models → Latent predictive world models
Abstract (arXiv)
Planning-time intervention fidelity is a distinct, measurable property of a learned world model: whether the model's own open-loop transitions move task variables the way matched environment interventions do. In the settings we test, it is neither revealed by reward fit nor ensured by task-anchored training. Across released TD-MPC2 checkpoint sizes, episode return falls as an operator-error diagnostic on task observables grows, while reward-prediction error stays small and nearly flat, and a self-supervised world model trained without task signal preserves the same operator substantially better than a task-anchored model on the shared task. A capture-gated matched-intervention audit then localizes what fails. On Cheetah, three LeWorldModel checkpoints capture the current task query and support decodable real intervention effects; however, their imagined five-step effects are worse than predicting no effect and worse than an environment-endpoint oracle. The failure is task-direction rotation with excess gain, not feature collapse. This severe pattern is conditional: five PreJEPA seeds retain an oracle-relative deficit without it, Finger Spin experiments extend the deficit beyond locomotion with heterogeneous severity across seeds, and shared-bank effect geometry is both candidate- and support-dependent. We also test practice-side questions. In DreamerV3 the posterior distribution, not its sample, carries the current query; ensemble disagreement ranks error only near training support; and a frozen support-aware score degrades held-out error ranking in both tested transfer directions while native disagreement remains informative in both. We conclude that intervention fidelity must be audited directly, capture-first, on the model's native interface.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Latent predictive world models, one of 11 directions in World models. Builds on 39 of its 51 references that are in the map. First posted on arXiv 30 Aug 2026.
Builds on
39 of 51 references.
Closest ancestors · 5
Same direction, newest first
- 2026LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels13 Mar
- 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planning7 Nov
- 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture19 Jan
- 2022Temporal Difference Learning for Model Predictive Control9 Mar
- 2018Learning Latent Dynamics for Planning from Pixels12 Nov
Other references · 34
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
- 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
- 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,485
- 2016Understanding intermediate layers using linear classifier probescited by 1,353
- 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
- 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
- 2023Mastering Diverse Domains through World Modelscited by 1,253
- 2020Mastering Atari with Discrete World Modelscited by 1,007
- 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
- 2018Recurrent World Models Facilitate Policy Evolutioncited by 785
- 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
22 more
- 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
- 2024Genie: Generative Interactive Environmentscited by 747
- 2019Designing and Interpreting Probes with Control Taskscited by 640
- 2020MOPO: Model-based Offline Policy Optimizationcited by 636
- 2020MOReL : Model-Based Offline Reinforcement Learningcited by 590
- 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 512
- 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 431
- 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 319
- 2021Modern Koopman Theory for Dynamical Systemscited by 276
- 2018Model-Ensemble Trust-Region Policy Optimizationcited by 219
- 2016Linear predictors for nonlinear dynamical systems: Koopman operator meets model predictive controlcited by 140
- 2024Evaluating the World Model Implicit in a Generative Modelcited by 113
- 2020Objective Mismatch in Model-based Reinforcement Learningcited by 96
- 2020The Value Equivalence Principle for Model-Based Reinforcement Learningcited by 94
- 2016The Predictron: End-To-End Learning and Planningcited by 93
- 2021Revisiting Design Choices in Offline Model-Based Reinforcement Learningcited by 50
- 2016Self-Correcting Models for Model-Based Reinforcement Learningcited by 43
- 2021Proper Value Equivalencecited by 33
- 2024ACT-Bench: Towards Action Controllable World Models for Autonomous Drivingcited by 14
- 2026How Should World Models Be Evaluated for Embodied Decision-Making? A Decision-Making-Centric Positioncited by 10
- 2026What Do World Models Learn in RL? Probing Latent Representations in Learned Environment Simulatorscited by 5
- 2026Biased Dreams: Limitations to Epistemic Uncertainty Quantification in Latent Dynamics Modelscited by 5
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026RolloutFaith: Auditing Persistent Internal Interventions in Visual World Modelcited by 0
- 2026Beyond a single latent space: a dual-latent world model for long-horizon planningcited by 0
- 2026When Low Prediction Error Misleads Planning: Diagnosing Representation, Dynamics, and Decision Failures in Latent World Modelscited by 0
- 2026When Low Prediction Error Misleads Planning: Diagnosing Representation, Dynamics, and Decision Failures in Latent World Models4 Oct 2026
- 2026Beyond a single latent space: a dual-latent world model for long-horizon planning29 Sep 2026
- 2026RolloutFaith: Auditing Persistent Internal Interventions in Visual World Model29 Sep 2026
Similar, not cited
Close in wording, no citation either way
- 2026Action-Consequence Alignment for Reliable Planning and Self-Improving in Latent World Models
- 2026Compact but Moving: Intervention-Relevant Geometry in Recurrent World Models
- 2026One-Step Prediction Fidelity Is a Poor Proxy for Planning Performance in JEPA World Models
- 2026verdi: retrieval is not transfer for continual world model optimization
- 2026Beyond Task Success: Stage-Wise Reliability of World Model Planning under Sensing Degradation
- 2026Building Comparative Motivation Profiles with Instrumental Interventions
- 2026World Models for Embodied Intelligence: From Plausible to Controllable to Actionable
- 2026Predicting Consequences and Reinforcing Navigation Policies with Latent World Models