mlsanityData to 14 Oct 2026

CausalVQA: A Physically Grounded Causal Reasoning Benchmark for Video Models

arXiv 2506.09943 ↗First posted 11 Jun 2025cs.CV cs.AIcited by 35

On the map: LLM reasoning → Multimodal reasoning

Abstract (arXiv)

We introduce CausalVQA, a benchmark dataset for video question answering (VQA) composed of question-answer pairs that probe models' understanding of causality in the physical world. Existing VQA benchmarks either tend to focus on surface perceptual understanding of real-world videos, or on narrow physical reasoning questions created using simulation environments. CausalVQA fills an important gap by presenting challenging questions that are grounded in real-world scenarios, while focusing on models' ability to predict the likely outcomes of different actions and events through five question types: counterfactual, hypothetical, anticipation, planning and descriptive. We designed quality control mechanisms that prevent models from exploiting trivial shortcuts, requiring models to base their answers on deep visual understanding instead of linguistic cues. We find that current frontier multimodal models fall substantially below human performance on the benchmark, especially on anticipation and hypothetical questions. This highlights a challenge for current systems to leverage spatial-temporal reasoning, understanding of physical principles, and comprehension of possible alternatives to make accurate predictions in real-world settings.

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 33 of its 46 references that are in the map. First posted on arXiv 11 Jun 2025.

Builds on

33 of 46 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling6 Dec
  2. 2023Perception Test: A Diagnostic Benchmark for Multimodal Video Models23 May
  3. 2022From Representation to Reasoning: Towards both Evidence and Commonsense Reasoning for Video Question-Answering30 May

Other references · 30

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2025Qwen2.5-VL Technical Reportcited by 5,587
  3. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  4. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  5. 2017The Kinetics Human Action Video Datasetcited by 1,909
  6. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  7. 2018Datasheets for Datasetscited by 1,386
  8. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 744
  9. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
  10. 2019ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answeringcited by 561
  11. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 490
  12. 2019CLEVRER: CoLlision Events for Video REpresentation and Reasoningcited by 484
18 more
  1. 2018TVQA: Localized, Compositional Video Question Answeringcited by 467
  2. 2024MM-LLMs: Recent Advances in MultiModal Large Language Modelscited by 278
  3. 2023Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questionscited by 263
  4. 2024VideoPhy: Evaluating Physical Commonsense for Video Generationcited by 228
  5. 2024Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generationcited by 174
  6. 2025Do generative video models understand physical principles?cited by 161
  7. 2019PHYRE: A New Benchmark for Physical Reasoningcited by 137
  8. 2024CinePile: A Long Video Question Answering Dataset and Benchmarkcited by 114
  9. 2024HourVideo: 1-Hour Video-Language Understandingcited by 113
  10. 2025PerceptionLM: Open-Access Data and Models for Detailed Visual Understandingcited by 92
  11. 2022Video Question Answering: Datasets, Algorithms and Challengescited by 84
  12. 2024A Survey on Benchmarks of Multimodal Large Language Modelscited by 82
  13. 2018IntPhys: A Framework and Benchmark for Visual Intuitive Physics Reasoningcited by 73
  14. 2022ComPhy: Compositional Physical Reasoning of Objects and Events from Videoscited by 63
  15. 2023Shot2Story: A New Benchmark for Comprehensive Understanding of Multi-shot Videoscited by 54
  16. 2024Distractor Generation in Multiple-Choice Tasks: A Survey of Methods, Datasets, and Evaluationcited by 25
  17. 2022CRIPP-VQA: Counterfactual Reasoning about Implicit Physical Properties via Video Question Answeringcited by 13
  18. 2023ACQUIRED: A Dataset for Answering Counterfactual Questions In Real-Life Videoscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 10; 2026: 25.

35 papers.

Sort:

By area:

  1. 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 165
  2. 2025Embodied AI Agents: Modeling the Worldcited by 61
  3. 2025PAI-Bench: A Comprehensive Benchmark For Physical AIcited by 34
  4. 2025From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Modelscited by 26
  5. 2026PhysicsMind: Sim and Real Mechanics Benchmarking for Physical Reasoning and Prediction in Foundational VLMs and World Modelscited by 11
  6. 2026How Should World Models Be Evaluated for Embodied Decision-Making? A Decision-Making-Centric Positioncited by 10
  7. 2026What-If World: A Causal Benchmark for General World Models in Embodied Scenarioscited by 8
  8. 2026CausalSpatial: A Benchmark for Object-Centric Causal Spatial Reasoningcited by 7
  9. 2025Counterfactual World Models via Digital Twin-conditioned Video Diffusioncited by 4
  10. 2026CRONOS: Benchmarking Counterfactual Physical Consistency in Video Modelscited by 4
  11. 2026OpenWorldLib: A Unified Codebase and Definition of Advanced World Modelscited by 3
  12. 2026YoCausal: How Far is Video Generation from World Model? A Causality Perspectivecited by 3
23 more
  1. 2026Causal Scaffolding for Physical Reasoning: A Benchmark for Causally-Informed Physical World Understanding in VLMscited by 3
  2. 2025SWITCH: Benchmarking Modeling and Handling of Tangible Interfaces in Long-horizon Embodied Scenarioscited by 2
  3. 2025Distilling Counterfactual Reasoning from Language to Vision: Causal Graph Guided Post-Training for Video Understandingcited by 2
  4. 2026PhysEditWorld: A Large-Scale Dataset Toward Physics-Editable World Modelscited by 2
  5. 2026Principia: Relational Physics Tests for Video Modelscited by 2
  6. 2025ChainReaction: Causal Chain-Guided Reasoning for Modular and Explainable Causal-Why Video Question Answeringcited by 1
  7. 2025PhysFire-WM: A Physics-Informed World Model for Emulating Fire Spread Dynamicscited by 1
  8. 2025Modeling the Mental World for Embodied AI: A Comprehensive Reviewcited by 1
  9. 2026Perception Without Engagement: Dissecting the Causal Discovery Deficit in LMMscited by 1
  10. 2026CometVLA: Co-Training on an Embodied Data Pyramid towards Physical Understandingcited by 1
  11. 2026MultiEcho: An Experimental Science of Learned Worldscited by 1
  12. 2026PhysVista: Benchmarking Physical Intelligence in VLMs via a Perception-Reasoning-Assessment Loopcited by 1
  13. 2025Spot The Ball: A Benchmark for Visual Social Inferencecited by 0
  14. 2026SCP: Spatial Causal Prediction in Videocited by 0
  15. 2026ICAT: Incident-Case-Grounded Adaptive Testing for Physical-Risk Prediction in Embodied World Modelscited by 0
  16. 2026Act2See: Emergent Active Visual Perception for Video Reasoningcited by 0
  17. 2026CaST-Bench: Benchmarking Causal Chain-Grounded Spatio-Temporal Reasoning for Video Question Answeringcited by 0
  18. 2026The Abstraction Gap in Vision-Language Causal Reasoningcited by 0
  19. 2026DMC-CF: Dynamic Multimodal CounterFactual QA benchmark for Causal Reasoningcited by 0
  20. 2026PhysAI-Bench: A Benchmark for LLM-Based Agentic Decision-Making in Autonomous UAV-Centric Physical AIcited by 0
  21. 2026Do World Models Make Better Robots? A Survey of Evaluation Benchmarks for Predictive Embodied Intelligencecited by 0
  22. 2026CCRV-Bench: Constraint-Based Evaluation of Causal Reasoning in Vision-Language Modelscited by 0
  23. 2026WOVEN: Weaving Visual World Modeling into Multimodal LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025CausalStep: A Benchmark for Explicit Stepwise Causal Reasoning in Videos
  2. 2025VCRBench: Exploring Long-form Causal Reasoning Capabilities of Large Video Language Models
  3. 2025LeAdQA: LLM-Driven Context-Aware Temporal Grounding for Video Question Answering
  4. 2025A Shortcut-aware Video-QA Benchmark for Physical Understanding via Minimal Video Pairs
  5. 2026UpstreamQA: A Modular Framework for Explicit Reasoning on Video Question Answering Tasks
  6. 2025Two Causally Related Needles in a Video Haystack
  7. 2025Pinpointing Trigger Moment for Grounded Video QA: Enhancing Spatio-temporal Grounding in Multimodal Large Language Models
  8. 2024Lost in Time: A New Temporal Benchmark for VideoLLMs