mlsanityData to 14 Oct 2026

CRAFT: A Benchmark for Causal Reasoning About Forces and inTeractions

arXiv 2012.04293 ↗First posted 8 Dec 2020cs.AI cs.CL cs.CVcited by 44

Not on the map. Why?

Abstract (arXiv)

Humans are able to perceive, understand and reason about causal events. Developing models with similar physical and causal understanding capabilities is a long-standing goal of artificial intelligence. As a step towards this direction, we introduce CRAFT, a new video question answering dataset that requires causal reasoning about physical forces and object interactions. It contains 58K video and question pairs that are generated from 10K videos from 20 different virtual environments, containing various objects in motion that interact with each other and the scene. Two question categories in CRAFT include previously studied descriptive and counterfactual questions. Additionally, inspired by the Force Dynamics Theory in cognitive linguistics, we introduce a new causal question category that involves understanding the causal interactions between objects through notions like cause, enable, and prevent. Our results show that even though the questions in CRAFT are easy for humans, the tested baseline models, including existing state-of-the-art methods, do not yet deal with the challenges posed in our benchmark.

Key facts. Cited by 44 papers in the map (data through 14 Oct 2026). First posted on arXiv 8 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 3; 2022: 4; 2023: 5; 2024: 11; 2025: 7; 2026: 14.

44 papers.

Sort:

By area:

  1. 2024How Far is Video Generation from World Model: A Physical Law Perspectivecited by 212
  2. 2025Do generative video models understand physical principles?cited by 161
  3. 2022ComPhy: Compositional Physical Reasoning of Objects and Events from Videoscited by 63
  4. 2023Physion++: Evaluating Physical Scene Understanding that Requires Online Inference of Different Physical Propertiescited by 54
  5. 2021Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learningcited by 53
  6. 2023ChatGPT Evaluation on Sentence Level Relations: A Focus on Temporal, Causal, and Discourse Relationscited by 40
  7. 2025Exploring the Evolution of Physics Cognition in Video Generation: A Surveycited by 31
  8. 2024ContPhy: Continuum Physical Concept Learning and Reasoning from Videoscited by 27
  9. 2024PhysGame: Uncovering Physical Commonsense Violations in Gameplay Videoscited by 22
  10. 2026Large Language Model Reasoning Failurescited by 21
  11. 2022A Survey on Machine Learning Approaches for Modelling Intuitive Physicscited by 20
  12. 2024InfiniBench: A Benchmark for Large Multi-Modal Models in Long-Form Movies and TV Showscited by 20
32 more
  1. 2024Compositional 4D Dynamic Scenes Understanding with Physics Priors for Video Question Answeringcited by 18
  2. 2024OSCaR: Object State Captioning and State Change Representationcited by 16
  3. 2025LikePhys: Evaluating Intuitive Physics Understanding in Video Diffusion Models via Likelihood Preferencecited by 16
  4. 2022CRIPP-VQA: Counterfactual Reasoning about Implicit Physical Properties via Video Question Answeringcited by 13
  5. 2024Compositional Physical Reasoning of Objects and Events from Videoscited by 12
  6. 2023CLEVRER-Humans: Describing Physical and Causal Events the Human Waycited by 11
  7. 2026PhysicsMind: Sim and Real Mechanics Benchmarking for Physical Reasoning and Prediction in Foundational VLMs and World Modelscited by 11
  8. 2024BenchAgents: Multi-Agent Systems for Structured Benchmark Creationcited by 9
  9. 2026PhysInOne: Visual Physics Learning and Reasoning in One Suitecited by 9
  10. 2026Physics-IQ Verifiedcited by 9
  11. 2021A Benchmark for Modeling Violation-of-Expectation in Physical Reasoning Across Event Categoriescited by 8
  12. 2023Neural Reasoning About Agents' Goals, Preferences, and Actionscited by 8
  13. 2023Benchmarks for Physical Reasoning AIcited by 8
  14. 2026What-If World: A Causal Benchmark for General World Models in Embodied Scenarioscited by 8
  15. 2025CAUSAL3D: A Comprehensive Benchmark for Causal Learning from Visual Datacited by 7
  16. 2024The Odyssey of Commonsense Causality: From Foundational Benchmarks to Cutting-Edge Reasoningcited by 5
  17. 2024MARPLE: A Benchmark for Long-Horizon Inferencecited by 5
  18. 2026TurkBench: A Benchmark for Evaluating Turkish Large Language Modelscited by 5
  19. 2021AVoE: A Synthetic 3D Dataset on Understanding Violation of Expectation for Artificial Cognitioncited by 4
  20. 2025CausalVerse: Benchmarking Causal Representation Learning with Configurable High-Fidelity Simulationscited by 4
  21. 2026CRONOS: Benchmarking Counterfactual Physical Consistency in Video Modelscited by 4
  22. 2024One-shot Video Imitation via Parameterized Symbolic Abstraction Graphscited by 3
  23. 2026Deltaynamics: Language-Based Representation for Inferring Rigid-Body Dynamics From Videoscited by 3
  24. 2026YoCausal: How Far is Video Generation from World Model? A Causality Perspectivecited by 3
  25. 2025InPhyRe Discovers: Large Multimodal Models Struggle in Inductive Physical Reasoningcited by 2
  26. 2026IMBench: A Benchmark for Intuitive Robotic Manipulationcited by 1
  27. 2022Learning to reason about and to act on physical cascading eventscited by 0
  28. 2025Think before You Simulate: Symbolic Reasoning to Orchestrate Neural Computation for Counterfactual Question Answeringcited by 0
  29. 2026ChronoPhyBench: Do MLLMs Truly Understand the World or Merely Exploit Language Priors?cited by 0
  30. 2026Bridging Physical Reasoning and Task Generalization via Visual Action Outcome Reasoning Alignmentcited by 0
  31. 2026What's the Catch? Evaluating Temporal Consistency in Vision-Language Modelscited by 0
  32. 2026Do World Models Make Better Robots? A Survey of Evaluation Benchmarks for Predictive Embodied Intelligencecited by 0