mlsanityData to 14 Oct 2026

Learning Physical Intuition of Block Towers by Example

arXiv 1603.01312 ↗First posted 3 Mar 2016cs.AIcited by 68

Not on the map. Why?

Abstract (arXiv)

Wooden blocks are a common toy for infants, allowing them to develop motor skills and gain intuition about the physical behavior of the world. In this paper, we explore the ability of deep feed-forward models to learn such intuitive physics. Using a 3D game engine, we create small towers of wooden blocks whose stability is randomized and render them collapsing (or remaining upright). This data allows us to train large convolutional network models which can accurately predict the outcome, as well as estimating the block trajectories. The models are also able to generalize in two important ways: (i) to new physical scenarios, e.g. towers with an additional block and (ii) to images of real wooden blocks, where it obtains a performance comparable to human subjects.

Key facts. Cited by 68 papers in the map (data through 14 Oct 2026). First posted on arXiv 3 Mar 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 1; 2021: 16; 2022: 13; 2023: 10; 2024: 9; 2025: 6; 2026: 12.

68 papers.

Sort:

By area:

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  2. 2021Physion: Evaluating Physical Prediction from Vision in Humans and Machinescited by 153
  3. 2022Mind's Eye: Grounded Language Model Reasoning through Simulationcited by 84
  4. 2021Learning to Compose Visual Relationscited by 72
  5. 2021Beyond Pick-and-Place: Tackling Robotic Stacking of Diverse Shapescited by 71
  6. 2021Grounding Physical Concepts of Objects and Events Through Dynamic Visual Reasoningcited by 68
  7. 2022ComPhy: Compositional Physical Reasoning of Objects and Events from Videoscited by 63
  8. 2021Dynamic Visual Reasoning by Learning Differentiable Physics Models from Video and Languagecited by 55
  9. 2022Learning Physical Dynamics with Subequivariant Graph Neural Networkscited by 39
  10. 2023Visual cognition in multimodal large language modelscited by 38
  11. 2021From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligencecited by 35
  12. 2024What is the Visual Cognition Gap between Humans and Multimodal LLMs?cited by 30
38 more
  1. 2023GRASP: A novel benchmark for evaluating language GRounding And Situated Physics understanding in multimodal language modelscited by 28
  2. 2021SwingBot: Learning Physical Features from In-hand Tactile Exploration for Dynamic Swing-up Manipulationcited by 24
  3. 2021CausalCity: Complex Simulations with Agency for Causal Discovery and Reasoningcited by 23
  4. 2024PhysGame: Uncovering Physical Commonsense Violations in Gameplay Videoscited by 22
  5. 2022A Survey on Machine Learning Approaches for Modelling Intuitive Physicscited by 20
  6. 2023I-PHYRE: Interactive Physical Reasoningcited by 15
  7. 2022Filtered-CoPhy: Unsupervised Learning of Counterfactual Physics in Pixel Spacecited by 14
  8. 2022Estimating counterfactual treatment outcomes over time in complex multiagent scenarioscited by 12
  9. 20233D-IntPhys: Towards More Generalized 3D-grounded Visual Intuitive Physics under Challenging Scenescited by 12
  10. 2024Compositional Physical Reasoning of Objects and Events from Videoscited by 12
  11. 2024InterDyn: Controllable Interactive Dynamics with Video Diffusion Modelscited by 12
  12. 2018OIL: Observational Imitation Learningcited by 11
  13. 2021Active Learning of Abstract Plan Feasibilitycited by 11
  14. 2022On the Learning Mechanisms in Physical Reasoningcited by 11
  15. 2025Visual Jenga: Discovering Object Dependencies via Counterfactual Inpaintingcited by 11
  16. 2024Physics Context Builders: A Modular Framework for Physical Reasoning in Vision-Language Modelscited by 10
  17. 2021PIP: Physical Interaction Prediction via Mental Simulation with Span Selectioncited by 8
  18. 2021A Benchmark for Modeling Violation-of-Expectation in Physical Reasoning Across Event Categoriescited by 8
  19. 2024FORCE: Physics-aware Human-object Interactioncited by 7
  20. 2025PhyBlock: A Progressive Benchmark for Physical Understanding and Planning via 3D Block Assemblycited by 7
  21. 2021Object and Relation Centric Representations for Push Effect Predictioncited by 5
  22. 2021Object-centric Video Prediction without Annotationcited by 5
  23. 2021AVoE: A Synthetic 3D Dataset on Understanding Violation of Expectation for Artificial Cognitioncited by 4
  24. 2025Testing the Limits of Fine-Tuning for Improving Visual Cognition in Vision Language Modelscited by 4
  25. 2026SPEAR: A Simulator for Photorealistic Embodied AI Researchcited by 4
  26. 2022Detecting and Accommodating Novel Types and Concepts in an Embodied Simulation Environmentcited by 3
  27. 2023Inferring Fluid Dynamics via Inverse Renderingcited by 3
  28. 2023Neuro-symbolic model for cantilever beams damage detectioncited by 3
  29. 2023X-VoE: Measuring eXplanatory Violation of Expectation in Physical Eventscited by 3
  30. 2023The Acquisition of Physical Knowledge in Generative Neural Networkscited by 3
  31. 2026Image2Garment: Simulation-ready Garment Generation from a Single Imagecited by 3
  32. 2021MoDeRNN: Towards Fine-grained Motion Details for Spatiotemporal Predictive Learningcited by 2
  33. 2022Exploiting Embodied Simulation to Detect Novel Object Classes Through Interactioncited by 2
  34. 20236-DoF Stability Field via Diffusion Modelscited by 2
  35. 2025InPhyRe Discovers: Large Multimodal Models Struggle in Inductive Physical Reasoningcited by 2
  36. 2026Can Vision Language Models Learn Intuitive Physics from Interaction?cited by 2
  37. 2022Recognising Affordances in Predicted Futures to Plan with Consideration of Non-canonical Affordance Effectscited by 1
  38. 2023USTEP: Spatio-Temporal Predictive Learning under A Unified Viewcited by 1

50 most cited of 68 All 68 (JSON)