# A Comparative Evaluation of Approximate Probabilistic Simulation and Deep Neural Networks as Accounts of Human Physical Scene Understanding

arXiv: https://arxiv.org/abs/1605.01138 · HTML: https://mlsanity.com/paper/1605.01138/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 May 2016.

## Built on by (most cited first, 9 of 9)

- [Physion: Evaluating Physical Prediction from Vision in Humans and Machines](https://mlsanity.com/paper/2106.08261/) (2021)
- [Visual Representation Learning Does Not Generalize Strongly Within the Same Domain](https://mlsanity.com/paper/2107.08221/) (2021)
- [3D-IntPhys: Towards More Generalized 3D-grounded Visual Intuitive Physics under Challenging Scenes](https://mlsanity.com/paper/2304.11470/) (2023)
- [A Benchmark for Modeling Violation-of-Expectation in Physical Reasoning Across Event Categories](https://mlsanity.com/paper/2111.08826/) (2021)
- [AVoE: A Synthetic 3D Dataset on Understanding Violation of Expectation for Artificial Cognition](https://mlsanity.com/paper/2110.05836/) (2021)
- [The Acquisition of Physical Knowledge in Generative Neural Networks](https://mlsanity.com/paper/2310.19943/) (2023)
- [Combining Commonsense Reasoning and Knowledge Acquisition to Guide Deep Learning in Robotics](https://mlsanity.com/paper/2201.10266/) (2022)
- [Stochastic Gradient Descent Captures How Children Learn About Physics](https://mlsanity.com/paper/2209.12344/) (2022)
- [Overhang Tower: Resource-Rational Adaptation in Sequential Physical Planning](https://mlsanity.com/paper/2604.09072/) (2026)

## Abstract (arXiv)

Humans demonstrate remarkable abilities to predict physical events in complex scenes. Two classes of models for physical scene understanding have recently been proposed: "Intuitive Physics Engines", or IPEs, which posit that people make predictions by running approximate probabilistic simulations in causal mental models similar in nature to video-game physics engines, and memory-based models, which make judgments based on analogies to stored experiences of previously encountered scenes and physical outcomes. Versions of the latter have recently been instantiated in convolutional neural network (CNN) architectures. Here we report four experiments that, to our knowledge, are the first rigorous comparisons of simulation-based and CNN-based models, where both approaches are concretely instantiated in algorithms that can run on raw image inputs and produce as outputs physical judgments such as whether a stack of blocks will fall. Both approaches can achieve super-human accuracy levels and can quantitatively predict human judgments to a similar degree, but only the simulation-based models generalize to novel situations in ways that people do, and are qualitatively consistent with systematic perceptual illusions and judgment asymmetries that people show.