# CRAFT: A Benchmark for Causal Reasoning About Forces and inTeractions

arXiv: https://arxiv.org/abs/2012.04293 · HTML: https://mlsanity.com/paper/2012.04293/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Dec 2020.

## Built on by (most cited first, 30 of 44)

- [How Far is Video Generation from World Model: A Physical Law Perspective](https://mlsanity.com/paper/2411.02385/) (2024)
- [Do generative video models understand physical principles?](https://mlsanity.com/paper/2501.09038/) (2025)
- [ComPhy: Compositional Physical Reasoning of Objects and Events from Videos](https://mlsanity.com/paper/2205.01089/) (2022)
- [Physion++: Evaluating Physical Scene Understanding that Requires Online Inference of Different Physical Properties](https://mlsanity.com/paper/2306.15668/) (2023)
- [Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learning](https://mlsanity.com/paper/2107.00848/) (2021)
- [ChatGPT Evaluation on Sentence Level Relations: A Focus on Temporal, Causal, and Discourse Relations](https://mlsanity.com/paper/2304.14827/) (2023)
- [Exploring the Evolution of Physics Cognition in Video Generation: A Survey](https://mlsanity.com/paper/2503.21765/) (2025)
- [ContPhy: Continuum Physical Concept Learning and Reasoning from Videos](https://mlsanity.com/paper/2402.06119/) (2024)
- [PhysGame: Uncovering Physical Commonsense Violations in Gameplay Videos](https://mlsanity.com/paper/2412.01800/) (2024)
- [Large Language Model Reasoning Failures](https://mlsanity.com/paper/2602.06176/) (2026)
- [A Survey on Machine Learning Approaches for Modelling Intuitive Physics](https://mlsanity.com/paper/2202.06481/) (2022)
- [InfiniBench: A Benchmark for Large Multi-Modal Models in Long-Form Movies and TV Shows](https://mlsanity.com/paper/2406.19875/) (2024)
- [Compositional 4D Dynamic Scenes Understanding with Physics Priors for Video Question Answering](https://mlsanity.com/paper/2406.00622/) (2024)
- [OSCaR: Object State Captioning and State Change Representation](https://mlsanity.com/paper/2402.17128/) (2024)
- [LikePhys: Evaluating Intuitive Physics Understanding in Video Diffusion Models via Likelihood Preference](https://mlsanity.com/paper/2510.11512/) (2025)
- [CRIPP-VQA: Counterfactual Reasoning about Implicit Physical Properties via Video Question Answering](https://mlsanity.com/paper/2211.03779/) (2022)
- [Compositional Physical Reasoning of Objects and Events from Videos](https://mlsanity.com/paper/2408.02687/) (2024)
- [CLEVRER-Humans: Describing Physical and Causal Events the Human Way](https://mlsanity.com/paper/2310.03635/) (2023)
- [PhysicsMind: Sim and Real Mechanics Benchmarking for Physical Reasoning and Prediction in Foundational VLMs and World Models](https://mlsanity.com/paper/2601.16007/) (2026)
- [BenchAgents: Multi-Agent Systems for Structured Benchmark Creation](https://mlsanity.com/paper/2410.22584/) (2024)
- [PhysInOne: Visual Physics Learning and Reasoning in One Suite](https://mlsanity.com/paper/2604.09415/) (2026)
- [Physics-IQ Verified](https://mlsanity.com/paper/2606.18943/) (2026)
- [A Benchmark for Modeling Violation-of-Expectation in Physical Reasoning Across Event Categories](https://mlsanity.com/paper/2111.08826/) (2021)
- [Neural Reasoning About Agents' Goals, Preferences, and Actions](https://mlsanity.com/paper/2312.07122/) (2023)
- [Benchmarks for Physical Reasoning AI](https://mlsanity.com/paper/2312.10728/) (2023)
- [What-If World: A Causal Benchmark for General World Models in Embodied Scenarios](https://mlsanity.com/paper/2605.27589/) (2026)
- [CAUSAL3D: A Comprehensive Benchmark for Causal Learning from Visual Data](https://mlsanity.com/paper/2503.04852/) (2025)
- [The Odyssey of Commonsense Causality: From Foundational Benchmarks to Cutting-Edge Reasoning](https://mlsanity.com/paper/2406.19307/) (2024)
- [MARPLE: A Benchmark for Long-Horizon Inference](https://mlsanity.com/paper/2410.01926/) (2024)
- [TurkBench: A Benchmark for Evaluating Turkish Large Language Models](https://mlsanity.com/paper/2601.07020/) (2026)

## Abstract (arXiv)

Humans are able to perceive, understand and reason about causal events. Developing models with similar physical and causal understanding capabilities is a long-standing goal of artificial intelligence. As a step towards this direction, we introduce CRAFT, a new video question answering dataset that requires causal reasoning about physical forces and object interactions. It contains 58K video and question pairs that are generated from 10K videos from 20 different virtual environments, containing various objects in motion that interact with each other and the scene. Two question categories in CRAFT include previously studied descriptive and counterfactual questions. Additionally, inspired by the Force Dynamics Theory in cognitive linguistics, we introduce a new causal question category that involves understanding the causal interactions between objects through notions like cause, enable, and prevent. Our results show that even though the questions in CRAFT are easy for humans, the tested baseline models, including existing state-of-the-art methods, do not yet deal with the challenges posed in our benchmark.