mlsanityData to 7 Oct 2026

IMPACT: Attention Is the Interaction Map for Scalable Interaction-Aware World Model Training

arXiv 2609.00161 ↗First posted 31 Aug 2026cs.AI cs.ROcited by 1

On the map: World models → Embodied and robot world models

Abstract (arXiv)

World models have made remarkable progress in action-conditioned future prediction for embodied agents, yet still struggle to model physically plausible interactions. Existing approaches address this limitation by constraining the generation process with external representations encoding motion, geometry, or semantics. Obtaining these spatiotemporally dense representations typically requires auxiliary estimators or manual annotations, limiting training scalability. We instead revisit the training objective and identify a supervision-allocation mismatch under the globally averaged mean squared error (MSE) denoising objective: prevalent static content dominates the optimization signal, leaving sparse dynamic-object regions critical to interaction generation disproportionately under-supervised. Motivated by this observation, we introduce IMPACT, a scalable Interaction-aware Model training framework with Prior-guided Attention Calibration and Targeting. IMPACT uses cross-attention associated with manipulated-object tokens as an internal spatiotemporal prior for action-conditioned changes. It samples candidate regions from this prior, calibrates them with detached local prediction errors to construct an interaction map, and uses the map to reweight denoising supervision, requiring neither external representations nor inference-time modifications. Extensive experiments on robot-arm and human-hand manipulation, spanning diverse control modalities and DiT backbones, show that IMPACT consistently outperforms the corresponding MSE-trained baselines, improving interaction fidelity, physical plausibility, and visual quality.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 33 of its 45 references that are in the map. First posted on arXiv 31 Aug 2026.

Builds on

33 of 45 references.

Closest ancestors · 8

Same direction, newest first

  1. 2026WorldScape Policy 2.0: Empowering Steerable World Action Modeling with Reasoning-Augmented Memory and In-Context Learning21 Jul
  2. 2026Worldscape-MoE: A Unified Mixture-of-Experts World Model for Scalable Heterogeneous Action Control4 Jul
  3. 2026Qwen-RobotWorld Technical Report: Unifying Embodied World Modeling through Language-Conditioned Video Generation15 Jun
  4. 2026EA-WM: Event-Aware Generative World Model with Structured Kinematic-to-Visual Action Fields7 May
  5. 2026STARRY: Spatial-Temporal Action-Centric World Modeling for Robotic Manipulation29 Apr
  6. 2026LOME: Learning Human-Object Manipulation with Action-Conditioned Egocentric World Model28 Mar
  7. 2026Hand2World: Autoregressive Egocentric Interaction Generation via Free-Space Hand Gestures10 Feb
  8. 2025Geometry-aware 4D Video Generation for Robot Manipulation1 Jul

Other references · 25

  1. 2024Qwen2.5 Technical Reportcited by 4,713
  2. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  3. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
  4. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  5. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  6. 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
  7. 2024Genie: Generative Interactive Environmentscited by 706
  8. 2025RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulationcited by 477
  9. 2025VACE: All-in-One Video Creation and Editingcited by 467
  10. 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 191
  11. 2025Matrix-game 2.0: An open-source real-time and streaming interactive world modelcited by 174
  12. 2025HunyuanVideo 1.5 Technical Reportcited by 148
13 more
  1. 2024Pandora: Towards General World Model with Natural Language Actions and Video Statescited by 83
  2. 2026WorldArena: A Unified Benchmark for Evaluating Perception and Functional Utility of Embodied World Modelscited by 68
  3. 2026iWorld-Bench: A Benchmark for Interactive World Models with a Unified Action Generation Frameworkcited by 14
  4. 2026WorldVLN: Autoregressive World Action Model for Aerial Vision-Language Navigationcited by 14
  5. 2026Multiplayer Interactive World Models with Representation Autoencoderscited by 14
  6. 2025AirScape: An Aerial Generative World Model with Motion Controllabilitycited by 7
  7. 2025Robotic VLA Benefits from Joint Learning with Motion Image Diffusioncited by 7
  8. 2026PhysisForcing: Physics Reinforced World Simulator for Robotic Manipulationcited by 7
  9. 2025Open-world Hand-Object Interaction Video Generation Based on Structure and Contact-aware Representationcited by 5
  10. 2026AlayaWorld: Long-Horizon and Playable Video World Generationcited by 4
  11. 2026Latent Temporal Discrepancy as Motion Prior: A Loss-Weighting Strategy for Dynamic Fidelity in T2Vcited by 3
  12. 2026CoInteract: Physically-Consistent Human-Object Interaction Video Synthesis via Spatially-Structured Co-Generationcited by 2
  13. 2026Lifting Embodied World Models for Planning and Controlcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Foresight Without Seeing: Latent Futures for World Action Models
  2. 2026Learning Transferable Dynamics Priors from Action to World Modeling
  3. 2026CAER: Causal Action Effect Reweighting for World Model Training
  4. 2026LDA-1B: Scaling Latent Dynamics Action Model via Universal Embodied Data Ingestion
  5. 2026Interactive World Simulator for Robot Policy Training and Evaluation
  6. 2026InternW0-Δ: A World Action Model Bridging Predictive Dynamics and Actions with 20K+ Hours of Open Data
  7. 2026AIM: Intent-Aware Unified world action Modeling with Spatial Value Maps
  8. 2026μ0: A Scalable 3D Interaction-Trace World Model