mlsanityData to 14 Oct 2026

Thinking Ahead: Foresight Intelligence in MLLMs and World Model

arXiv 2511.18735 ↗First posted 24 Nov 2025cs.CV cs.AIcited by 0

On the map: World models → World models for autonomous driving

also in 1 more
Abstract (arXiv)

In this work, we define Foresight Intelligence as the capability to anticipate and interpret future events-an ability essential for applications such as autonomous driving, yet largely overlooked by existing research. To bridge this gap, we introduce FSU-QA, a new Visual Question-Answering (VQA) dataset specifically designed to elicit and evaluate Foresight Intelligence. Using FSU-QA, we conduct the first comprehensive study of state-of-the-art Vision-Language Models (VLMs) under foresight-oriented tasks, revealing that current models still struggle to reason about future situations. Beyond serving as a benchmark, FSU-QA also enables the assessment of world models by measuring the semantic coherence of their generated predictions, quantified through performance gains when VLMs are augmented with such outputs. Our experiments further demonstrate that FSU-QA can effectively enhance foresight reasoning: even small VLMs fine-tuned on FSU-QA surpass much larger, advanced models by a substantial margin. Together, these findings position FSU-QA as a principled foundation for developing next-generation models capable of truly anticipating and understanding future events. Furthermore, beyond model performance, we examine whether WM-generated predictions remain semantically consistent by using VLM-based proxy judges, and validate this evaluation protocol through shuffled control experiments. Fine-tuning models on FSU-QA leads to substantial improvements in foresight understanding, demonstrating the dataset's effectiveness and offering a principled foundation for future research.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 40 of its 55 references that are in the map. First posted on arXiv 24 Nov 2025.

Builds on

40 of 55 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Epona: Autoregressive Diffusion World Model for Autonomous Driving30 Jun
  2. 2025OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoning6 Apr
  3. 2024DrivingWorld: Constructing World Model for Autonomous Driving via Video GPT27 Dec
  4. 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation11 Mar
  5. 2023DriveDreamer: Towards Real-world-driven World Models for Autonomous Driving18 Sep
  6. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles22 Jun

Other references · 34

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  2. 2023Visual Instruction Tuningcited by 7,869
  3. 2025Qwen2.5-VL Technical Reportcited by 5,587
  4. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  5. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  6. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
  7. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  8. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  9. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
  10. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
  11. 2024Genie: Generative Interactive Environmentscited by 747
  12. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 744
22 more
  1. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 658
  2. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 611
  3. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 536
  4. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Modelcited by 461
  5. 2023GPT-Driver: Learning to Drive with GPTcited by 347
  6. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 346
  7. 2023NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenariocited by 262
  8. 2024Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understandingcited by 213
  9. 2025MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligencecited by 183
  10. 2023Panacea: Panoramic and Controllable Video Generation for Autonomous Drivingcited by 168
  11. 2023A Language Agent for Autonomous Drivingcited by 164
  12. 2025SimLingo: Vision-Only Closed-Loop Autonomous Driving with Language-Action Alignmentcited by 150
  13. 20243DSRBench: A Comprehensive 3D Spatial Reasoning Benchmarkcited by 139
  14. 2023Equivariant Multi-Modality Image Fusioncited by 99
  15. 2024SeeGround: See and Ground for Zero-Shot Open-Vocabulary 3D Visual Groundingcited by 67
  16. 2025VLM4D: Towards Spatiotemporal Awareness in Vision Language Modelscited by 46
  17. 2024Orient Anything: Learning Robust Object Orientation Estimation from Rendering 3D Modelscited by 45
  18. 2025DeepVerse: 4D Autoregressive Video Generation as a World Modelcited by 40
  19. 2025DSI-Bench: A Benchmark for Dynamic Spatial Intelligencecited by 23
  20. 2025Unifying 2D and 3D Vision-Language Understandingcited by 15
  21. 2025DriveGEN: Generalized and Robust 3D Detection in Driving via Controllable Text-to-Image Diffusion Generationcited by 13
  22. 2025Future-Aware Interaction Network For Motion Forecastingcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026DeepSight: Long-Horizon World Modeling via Latent States Prediction for End-to-End Autonomous Driving
  2. 2023Merlin:Empowering Multimodal LLMs with Foresight Minds
  3. 2026From Sight to Foresight: Predictive Spatial Reasoning in Vision-Language Models
  4. 2026ForeAct: Steering Your VLA with Efficient Visual Foresight Planning
  5. 2026Current Agents Fail to Leverage World Model as Tool for Foresight
  6. 2025CoFFT: Chain of Foresight-Focus Thought for Visual Language Models
  7. 2025More Than Meets the Eye? Uncovering the Reasoning-Planning Disconnect in Training Vision-Language Driving Models
  8. 2025FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Driving