Reimagination with Test-time Observation Interventions: Distractor-Robust World Model Predictions for Visual Model Predictive Control
On the map: World models → Embodied and robot world models
Abstract (arXiv)
World models enable robots to "imagine" future observations given current observations and planned actions, and have been increasingly adopted as generalized dynamics models to facilitate robot learning. Despite their promise, these models remain brittle when encountering novel visual distractors such as objects and background elements rarely seen during training. Specifically, novel distractors can corrupt action outcome predictions, causing downstream failures when robots rely on the world model imaginations for planning or action verification. In this work, we propose Reimagination with Observation Intervention (ReOI), a simple yet effective test-time strategy that enables world models to predict more reliable action outcomes in open-world scenarios where novel and unanticipated visual distractors are inevitable. Given the current robot observation, ReOI first detects visual distractors by identifying which elements of the scene degrade in physically implausible ways during world model prediction. Then, it modifies the current observation to remove these distractors and bring the observation closer to the training distribution. Finally, ReOI "reimagines" future outcomes with the modified observation and reintroduces the distractors post-hoc to preserve visual consistency for downstream planning and verification. We validate our approach on a suite of robotic manipulation tasks in the context of action verification, where the verifier needs to select desired action plans based on predictions from a world model. Our results show that ReOI is robust to both in-distribution and out-of-distribution visual distractors. Notably, it improves task success rates by up to 3x in the presence of novel distractors, significantly outperforming action verification that relies on world model predictions without imagination interventions.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 23 of its 28 references that are in the map. First posted on arXiv 19 Jun 2025.
Builds on
23 of 28 references.
Closest ancestors · 3
Same direction, newest first
- 2024FLIP: Flow-Centric Generative Planning as General-Purpose Manipulation World Model11 Dec
- 2024AD3: Implicit Action is the Key for World Models to Distinguish the Diverse Visual Distractors15 Mar
- 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusion7 Mar
Other references · 20
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
- 2018Recurrent World Models Facilitate Policy Evolutioncited by 785
- 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 436
- 2022DayDreamer: World Models for Physical Robot Learningcited by 400
- 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planningcited by 386
- 2022Denoised MDPs: Learning World Models Better Than the World Itselfcited by 68
- 2021Learning Task Informed Abstractionscited by 58
- 2025From Foresight to Forethought: VLM-In-the-Loop Policy Steering via Latent Alignmentcited by 52
- 2024RoVi-Aug: Robot and Viewpoint Augmentation for Cross-Embodiment Robot Learningcited by 46
8 more
- 2024Run-time Observation Interventions Make Vision-Language-Action Models More Visually Robustcited by 33
- 2020Planning with Learned Dynamics: Probabilistic Guarantees on Safety and Reachability via Lipschitz Constantscited by 25
- 2023Transferring Foundation Models for Generalizable Robotic Manipulationcited by 18
- 2023RePo: Resilient Model-Based Reinforcement Learning by Regularizing Posterior Predictabilitycited by 18
- 2025Improving Transformer World Models for Data-Efficient RLcited by 13
- 2023SeMAIL: Eliminating Distractors in Visual Imitation via Separated Modelscited by 10
- 2023ROSO: Improving Robotic Policy Inference via Synthetic Observationscited by 3
- 2023Less is more -- the Dispatcher/ Executor principle for multi-task Reinforcement Learningcited by 2
Built on by
Citing papers per year: 2025: 2; 2026: 4.
6 papers.
- 2025A Comprehensive Survey on World Models for Embodied AIcited by 68
- 2025Counterfactual VLA: Self-Reflective Vision-Language-Action Model with Adaptive Reasoningcited by 22
- 2026TEXEDO : Test Time Scaling for Controller-aware Language-conditioned Humanoid Motion Generationcited by 3
- 2026Diagnosing JEPA World Models with Action-Conditioned Predictive Consistencycited by 3
- 2026World Models for Embodied Intelligence: From Plausible to Controllable to Actionablecited by 1
- 2026Dynamic Manipulation with World-Action Models via Counterfactual Planningcited by 0
- 2026Dynamic Manipulation with World-Action Models via Counterfactual Planning27 Sep 2026
- 2026World Models for Embodied Intelligence: From Plausible to Controllable to Actionable15 Sep 2026
- 2026Diagnosing JEPA World Models with Action-Conditioned Predictive Consistency13 Aug 2026
- 2026TEXEDO : Test Time Scaling for Controller-aware Language-conditioned Humanoid Motion Generation22 Jun 2026
- 2025Counterfactual VLA: Self-Reflective Vision-Language-Action Model with Adaptive Reasoning30 Dec 2025
- 2025A Comprehensive Survey on World Models for Embodied AI19 Oct 2025
Similar, not cited
Close in wording, no citation either way
- 2026When to Trust Imagination: Adaptive Action Execution for World Action Models
- 2026Foresight Without Seeing: Latent Futures for World Action Models
- 2026DELE-w0.5: Inferring Action from Future Latent State for Robotic Manipulation
- 2026Feedback World Model Enables Precise Guidance of Diffusion Policy
- 2026Bridge-WA: Predicting Where and How the World Changes for Robotic Action
- 2026GlanceWAM: Sparse Test-Time Imagination for World-Action Models
- 2026Masked Visual Actions for Unified World Modeling
- 2026Remembering What Matters for Long-Horizon Robotic World Models