mlsanityData to 14 Oct 2026

FutureDuet: Decoupling Observation Access from Future Supervision in World Action Models

arXiv 2609.34362 ↗First posted 28 Sep 2026cs.ROcited by 0

On the map: World models → Embodied and robot world models

Abstract (arXiv)

World Action Models (WAMs) augment robot action generation with future visual supervision. Existing WAMs commonly fuse main and wrist observations into one visual stream and train both with the same future-video objective, despite their different visual dynamics. A stable main camera reveals scene-level task evolution, whereas wrist cameras move with the end effector, mixing local interaction changes with viewpoint shifts and self-occlusion. These contrasting predictive demands suggest that the two views may benefit from different future objectives. We introduce FutureDuet, which retains both views for control, while allowing each visual stream to receive a different future objective. For the main view, future RGB models task evolution, while interaction masks and robot skeletons focus supervision on task objects and robot motion. For the wrist stream, future latent prediction models short-horizon interaction changes without requiring pixel-level reconstruction. ActionDiT jointly reads the resulting Task State and Interaction State, combining scene-level progress with close-range interaction evidence. All auxiliary prediction modules are training-only, adding no inference overhead. FutureDuet achieves 94.2% clean and 94.1% randomized success on RoboTwin50 and 99.2% average success on LIBERO. The improvements are most pronounced on six RoboTwin50 tasks that require precise interaction, averaging gains of 9.2% and 12.8% over Fast-WAM in clean and randomized settings. Controlled studies further show complementary gains from separating the wrist pathway and designing future supervision separately for the two views.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 35 of its 51 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

35 of 51 references.

Closest ancestors · 14

Same direction, newest first

  1. 2026Foresight Without Seeing: Latent Futures for World Action Models12 Aug
  2. 2026JEPA-WAM: Learning Vision-Language-Action Policies with Joint-Embedding World Modeling10 Aug
  3. 2026DreamWAM: Beyond RGB Future Prediction for World Action Models5 Aug
  4. 2026LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies14 Jun
  5. 2026MaskWAM: Unifying Mask Prompting and Prediction for World-Action Models11 Jun
  6. 2026Unified 4D World Action Modeling from Video Priors with Asynchronous Denoising29 Apr
  7. 2026Fast-WAM: Do World Action Models Need Test-time Future Imagination?17 Mar
  8. 2026VLA-JEPA: Enhancing Vision-Language-Action Model with Latent World Model10 Feb
  9. 2026Causal World Modeling for Robot Control29 Jan
  10. 2025Motus: A Unified Latent Action World Model15 Dec
  11. 2025RynnVLA-002: A Unified Vision-Language-Action and World Model21 Nov
  12. 2025Dual-Stream Diffusion for World-Model Augmented Vision-Language-Action Model31 Oct
2 more
  1. 2025WorldVLA: Towards Autoregressive Action World Model26 Jun
  2. 2025DreamGen: Unlocking Generalization in Robot Learning through Video World Models19 May

Other references · 21

  1. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  2. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  3. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,761
  4. 2024Octo: An Open-Source Generalist Robot Policycited by 1,484
  5. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,225
  6. 2025SAM 3: Segment Anything with Conceptscited by 906
  7. 2025X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Modelcited by 303
  8. 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 300
  9. 2024GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulationcited by 296
  10. 2025DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledgecited by 213
  11. 2024Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Modelscited by 163
  12. 2025mimic-video: Video-Action Models for Generalizable Robot Control Beyond VLAscited by 142
9 more
  1. 2026V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learningcited by 117
  2. 2026A Pragmatic VLA Foundation Modelcited by 103
  3. 2025Video Generators are Robot Policiescited by 92
  4. 2025ReconVLA: Reconstructive Vision-Language-Action Model as Effective Robot Perceivercited by 81
  5. 2026DiT4DiT: Jointly Modeling Video Dynamics and Actions for Generalizable Robot Controlcited by 76
  6. 2025Vidar: Embodied Video Diffusion Model for Generalist Manipulationcited by 62
  7. 2025Point Policy: Unifying Observations and Actions with Key Points for Robot Manipulationcited by 47
  8. 2026Disentangled Robot Learning via Separate Forward and Inverse Dynamics Pretrainingcited by 14
  9. 2026World-to-Wrist: Task-Conditioned Future Wrist Modeling for Fine-Grained Robot Manipulationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026DC-WAM: Dynamic-Centric Visual Supervision and Reasoning for World-Action Models
  2. 2026InternW0-Δ: A World Action Model Bridging Predictive Dynamics and Actions with 20K+ Hours of Open Data
  3. 2026Bridge-WA: Predicting Where and How the World Changes for Robotic Action
  4. 2026PACT-WAM: Predicting Actions and Visual Foresight with Compact Temporal Encoding for Robot Manipulation
  5. 2026DELE-w0.5: Inferring Action from Future Latent State for Robotic Manipulation
  6. 2026From World Models to World Action Models: Rethinking Next-State Prediction
  7. 2026MotionWeave: Learning Motion-Centered Future Dynamics for Vision-Language-Action Policies
  8. 2026MoWAM: Explicit Future Motion Prediction for Efficient World Action Models