mlsanityData to 14 Oct 2026

Reimagination with Test-time Observation Interventions: Distractor-Robust World Model Predictions for Visual Model Predictive Control

arXiv 2506.16565 ↗First posted 19 Jun 2025cs.RO cs.AI cs.CVcited by 6

On the map: World models → Embodied and robot world models

Abstract (arXiv)

World models enable robots to "imagine" future observations given current observations and planned actions, and have been increasingly adopted as generalized dynamics models to facilitate robot learning. Despite their promise, these models remain brittle when encountering novel visual distractors such as objects and background elements rarely seen during training. Specifically, novel distractors can corrupt action outcome predictions, causing downstream failures when robots rely on the world model imaginations for planning or action verification. In this work, we propose Reimagination with Observation Intervention (ReOI), a simple yet effective test-time strategy that enables world models to predict more reliable action outcomes in open-world scenarios where novel and unanticipated visual distractors are inevitable. Given the current robot observation, ReOI first detects visual distractors by identifying which elements of the scene degrade in physically implausible ways during world model prediction. Then, it modifies the current observation to remove these distractors and bring the observation closer to the training distribution. Finally, ReOI "reimagines" future outcomes with the modified observation and reintroduces the distractors post-hoc to preserve visual consistency for downstream planning and verification. We validate our approach on a suite of robotic manipulation tasks in the context of action verification, where the verifier needs to select desired action plans based on predictions from a world model. Our results show that ReOI is robust to both in-distribution and out-of-distribution visual distractors. Notably, it improves task success rates by up to 3x in the presence of novel distractors, significantly outperforming action verification that relies on world model predictions without imagination interventions.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 23 of its 28 references that are in the map. First posted on arXiv 19 Jun 2025.

Builds on

23 of 28 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024FLIP: Flow-Centric Generative Planning as General-Purpose Manipulation World Model11 Dec
  2. 2024AD3: Implicit Action is the Key for World Models to Distinguish the Diverse Visual Distractors15 Mar
  3. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusion7 Mar

Other references · 20

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  3. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
  4. 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
  5. 2018Recurrent World Models Facilitate Policy Evolutioncited by 785
  6. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 436
  7. 2022DayDreamer: World Models for Physical Robot Learningcited by 400
  8. 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planningcited by 386
  9. 2022Denoised MDPs: Learning World Models Better Than the World Itselfcited by 68
  10. 2021Learning Task Informed Abstractionscited by 58
  11. 2025From Foresight to Forethought: VLM-In-the-Loop Policy Steering via Latent Alignmentcited by 52
  12. 2024RoVi-Aug: Robot and Viewpoint Augmentation for Cross-Embodiment Robot Learningcited by 46
8 more
  1. 2024Run-time Observation Interventions Make Vision-Language-Action Models More Visually Robustcited by 33
  2. 2020Planning with Learned Dynamics: Probabilistic Guarantees on Safety and Reachability via Lipschitz Constantscited by 25
  3. 2023Transferring Foundation Models for Generalizable Robotic Manipulationcited by 18
  4. 2023RePo: Resilient Model-Based Reinforcement Learning by Regularizing Posterior Predictabilitycited by 18
  5. 2025Improving Transformer World Models for Data-Efficient RLcited by 13
  6. 2023SeMAIL: Eliminating Distractors in Visual Imitation via Separated Modelscited by 10
  7. 2023ROSO: Improving Robotic Policy Inference via Synthetic Observationscited by 3
  8. 2023Less is more -- the Dispatcher/ Executor principle for multi-task Reinforcement Learningcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 4.

6 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026When to Trust Imagination: Adaptive Action Execution for World Action Models
  2. 2026Foresight Without Seeing: Latent Futures for World Action Models
  3. 2026DELE-w0.5: Inferring Action from Future Latent State for Robotic Manipulation
  4. 2026Feedback World Model Enables Precise Guidance of Diffusion Policy
  5. 2026Bridge-WA: Predicting Where and How the World Changes for Robotic Action
  6. 2026GlanceWAM: Sparse Test-Time Imagination for World-Action Models
  7. 2026Masked Visual Actions for Unified World Modeling
  8. 2026Remembering What Matters for Long-Horizon Robotic World Models