mlsanityData to 14 Oct 2026

ForeTime-VLA: Causal Future-Token Distillation from a World Action Model for Conveyor-Belt Manipulation

arXiv 2608.20735 ↗First posted 21 Aug 2026cs.AI cs.ROcited by 0

On the map: World models → Embodied and robot world models

also in 1 more
Abstract (arXiv)

Manipulating moving objects requires a policy to anticipate contact events, yet vision-language-action (VLA) policies are commonly fine-tuned from the current observation alone. World action models (WAMs) learn predictive dynamics, but running a video-scale teacher or explicitly imagining future frames at deployment is costly. We introduce ForeTime-VLA, a dense pi0.5 policy that distills a future-aware, action-equivalent representation from a frozen Fast-WAM-derived teacher while remaining causal at inference. Offline, current and future video latents are compressed into a whitened 64-D target. Online, an eight-frame history encoder predicts this target together with manipulation phase and normalized time-to-transition. Four future tokens and one phase token condition the VLM prefix, while the predicted future and transition horizon condition the action expert. Training retains the original flow-matching action target and adds cosine, relational geometry, phase, time-to-transition, and action-equivalence objectives. On a deduplicated conveyor-belt dataset, we compare 40k-step checkpoints on 768 matched windows per split. Test MAE decreases from 0.134119 to 0.130593 (2.63%; paired-bootstrap 95% CI: 0.82-4.48% improvement), and test L2 decreases by 3.02%, at a 2.46-2.93% latency cost. In quantitative real-robot evaluation, ForeTime-VLA achieves 81.1% stationary and 58.9% slow-moving grasp success, exceeding the next-best reference by 12.2 and 22.2 percentage points, respectively. Across three belt speeds, it completes 44/90 grasps versus 23/90 for pi0.5, including 11/30 versus 2/30 at fast speed. The agreement between offline orientation gains and reduced real-robot contact-pose failures supports causal future-token distillation as an effective way to improve dynamic manipulation without deploying the world-model teacher.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 35 of its 37 references that are in the map. First posted on arXiv 21 Aug 2026.

Builds on

35 of 37 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026DECOWAM: Decoupled Whole-Body World-Action Model for Legged Mobile Manipulation20 Aug
  2. 2026Fast-WAM: Do World Action Models Need Test-time Future Imagination?17 Mar
  3. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusion7 Mar

Other references · 32

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  2. 2022Flow Matching for Generative Modelingcited by 5,702
  3. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  4. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  5. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  6. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  7. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  8. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  9. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,761
  10. 2014FitNets: Hints for Thin Deep Netscited by 1,653
  11. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  12. 2018World Modelscited by 1,513
20 more
  1. 2024Octo: An Open-Source Generalist Robot Policycited by 1,484
  2. 2023Mastering Diverse Domains through World Modelscited by 1,253
  3. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,225
  4. 2016Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfercited by 1,077
  5. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,057
  6. 2019Relational Knowledge Distillationcited by 889
  7. 2022A Generalist Agentcited by 808
  8. 2019Contrastive Representation Distillationcited by 748
  9. 2024Genie: Generative Interactive Environmentscited by 747
  10. 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 665
  11. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 584
  12. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 551
  13. 2025SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Roboticscited by 530
  14. 2023Learning Universal Policies via Text-Guided Video Generationcited by 522
  15. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 407
  16. 2021Understanding Dimensional Collapse in Contrastive Self-supervised Learningcited by 378
  17. 2023Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulationcited by 364
  18. 2024Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representationscited by 289
  19. 2016Deep Visual Foresight for Planning Robot Motioncited by 285
  20. 2018Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Controlcited by 230

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Intercepting the Future: Latent-Space Predictive World Model for Dynamic VLA Manipulation
  2. 2026InternW0-Δ: A World Action Model Bridging Predictive Dynamics and Actions with 20K+ Hours of Open Data
  3. 2026PACT-WAM: Predicting Actions and Visual Foresight with Compact Temporal Encoding for Robot Manipulation
  4. 2026WAVE-0: Bridging VLA Policies and WAMs with Lightweight Latent Future Prediction
  5. 2026GE-Act 2.0: Pretraining and Scaling a World-Action Model for Robotic Manipulation
  6. 2026Box-WAM: Box Trajectories as Compact State Representations for World Action Models
  7. 2026DELE-w0.5: Inferring Action from Future Latent State for Robotic Manipulation
  8. 2026Foresight Without Seeing: Latent Futures for World Action Models