X-Foresight: A Joint Vision-Action Causal Forecasting Network via Predictive World Modeling
On the map: World models → Embodied and robot world models
also in 1 more
Abstract (arXiv)
Physical world knowledge resides mainly in videos. Equipping Vision-Language-Action (VLA) models with such knowledge is fundamental for safe and generalizable planning. Predictive world modeling enables VLA to internalize physical dynamics and long-term causality by predicting future video from past observations. However, naive next-frame prediction faces two challenges: 1) unlike semantically distinct text tokens, video tokens are low-entropy and redundant, causing prediction to degenerate into trivial extrapolation. 2) world modeling poses a temporal dilemma: dense prediction captures instantaneous dynamics, but cannot efficiently model long-horizon causality. To learn world knowledge effectively, we introduce X-Foresight, a predictive world model integrated directly into the VLA architecture to jointly learn world modeling and real-time action control. At its core lies a long-horizon chunk-wise auto-regressive strategy that addresses both challenges: by predicting semantically distant chunks rather than adjacent frames, it escapes trivial extrapolation, while preserving dense intra-chunk frames for instantaneous dynamics and sparse inter-chunk transitions for long-term causality. A curriculum learning schedule progressively extends prediction horizons and stabilizes long-horizon training. To capture long-term causality effectively, we present temporal importance sampling, which concentrates supervision on safety-critical chunks identified by ego-motion and behavioral signals. We further delegate photorealistic synthesis to a diffusion-based multi-view renderer, improving photorealistic appearance. Comprehensive experiments demonstrate that X-Foresight significantly outperforms VLA baselines in planning performance while maintaining strong generative fidelity, establishing a robust paradigm for world-knowledge-driven autonomous systems.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 19 of its 22 references that are in the map. First posted on arXiv 24 May 2026.
Builds on
19 of 22 references.
References · 19
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
- 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
- 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,234
- 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 1,039
- 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
- 2024Genie: Generative Interactive Environmentscited by 747
7 more
- 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 625
- 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 433
- 2025Latent Diffusion Model without Variational Autoencodercited by 88
- 2025Radial Attention: O(nlog n) Sparse Attention with Energy Decay for Long Video Generationcited by 69
- 2026Helios: Real Real-Time Long Video Generation Modelcited by 65
- 2025One Layer Is Enough: Adapting Pretrained Visual Encoders for Image Generationcited by 28
- 2026X-World: Controllable Ego-Centric Multi-Camera World Models for Scalable End-to-End Drivingcited by 8
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Foresight Without Seeing: Latent Futures for World Action Models
- 2026PLaW-VLA: Predictive Latent World Modeling for Vision-Language-Action Policies
- 2026VIDEAS: Distilling Explicit Action Semantics from Demonstration Videos for World Models via Prior-Guided Simulation
- 2026World Tokens: Enhancing Embodied Policies with Training-Time World Modeling
- 2026WAVE-0: Bridging VLA Policies and WAMs with Lightweight Latent Future Prediction
- 2026Parallel Predictive World Models for Accurate and Efficient Long-Horizon Planning
- 2025From Forecasting to Planning: Policy World Model for Collaborative State-Action Prediction
- 2026Devol-ONE: One Autoregressive Mixture of Transformers to Unify Vision-Language-Action and Latent World Modeling