mlsanityData to 14 Oct 2026

X-VoE: Measuring eXplanatory Violation of Expectation in Physical Events

arXiv 2308.10441 ↗First posted 21 Aug 2023cs.AI cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Intuitive physics is pivotal for human understanding of the physical world, enabling prediction and interpretation of events even in infancy. Nonetheless, replicating this level of intuitive physics in artificial intelligence (AI) remains a formidable challenge. This study introduces X-VoE, a comprehensive benchmark dataset, to assess AI agents' grasp of intuitive physics. Built on the developmental psychology-rooted Violation of Expectation (VoE) paradigm, X-VoE establishes a higher bar for the explanatory capacities of intuitive physics models. Each VoE scenario within X-VoE encompasses three distinct settings, probing models' comprehension of events and their underlying explanations. Beyond model evaluation, we present an explanation-based learning system that captures physics dynamics and infers occluded object states solely from visual sequences, without explicit occlusion labels. Experimental outcomes highlight our model's alignment with human commonsense when tested against X-VoE. A remarkable feature is our model's ability to visually expound VoE events by reconstructing concealed scenes. Concluding, we discuss the findings' implications and outline future research directions. Through X-VoE, we catalyze the advancement of AI endowed with human-like intuitive physics capabilities.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 29 of its 46 references that are in the map. First posted on arXiv 21 Aug 2023.

Builds on

29 of 46 references.

References · 29

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  4. 2020Object-Centric Learning with Slot Attentioncited by 876
  5. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 392
  6. 2015Deep multi-scale video prediction beyond mean square errorcited by 300
  7. 2016Unsupervised Learning for Physical Interaction through Video Predictioncited by 260
  8. 2017Stochastic Variational Video Predictioncited by 234
  9. 2017Video Frame Synthesis using Deep Voxel Flowcited by 209
  10. 2016Deep Predictive Coding Networks for Video Prediction and Unsupervised Learningcited by 204
  11. 2020Disentangling Physical Dynamics from Unknown Factors for Unsupervised Video Predictioncited by 195
  12. 2019Scaling Autoregressive Video Modelscited by 190
17 more
  1. 2017Decomposing Motion and Content for Natural Video Sequence Predictioncited by 169
  2. 2021A Good Image Generator Is What You Need for High-Resolution Video Synthesiscited by 163
  3. 2017Unsupervised Learning of Disentangled Representations from Videocited by 139
  4. 2018Learning to Decompose and Disentangle Representations for Video Predictioncited by 121
  5. 2016A Compositional Object-Based Approach to Learning Physical Dynamicscited by 119
  6. 2019Spatial Broadcast Decoder: A Simple Architecture for Learning Disentangled Representations in VAEscited by 117
  7. 2017Visual Interaction Networkscited by 95
  8. 2018IntPhys: A Framework and Benchmark for Visual Intuitive Physics Reasoningcited by 73
  9. 2016Learning Physical Intuition of Block Towers by Examplecited by 68
  10. 2021Grounding Physical Concepts of Objects and Events Through Dynamic Visual Reasoningcited by 68
  11. 2020Attention over learned object embeddings enables complex visual reasoningcited by 59
  12. 2021Dynamic Visual Reasoning by Learning Differentiable Physics Models from Video and Languagecited by 55
  13. 2018SDCNet: Video Prediction Using Spatially-Displaced Convolutioncited by 50
  14. 2021The Emergence of Objectness: Learning Zero-Shot Segmentation from Videoscited by 45
  15. 2018Probing Physics Knowledge Using Tools from Developmental Psychologycited by 12
  16. 2020Occlusion resistant learning of intuitive physics from videoscited by 8
  17. 2021A Benchmark for Modeling Violation-of-Expectation in Physical Reasoning Across Event Categoriescited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2026Learning Physics-Grounded 4D Dynamics with Neural Gaussian Force Fieldscited by 6
  2. 2025Neural Force Field: Few-shot Learning of Generalized Physical Reasoningcited by 1
  3. 2026FADE: From Passive Verification to Active Discovery in Counterfactual Video Understandingcited by 0