mlsanityData to 7 Oct 2026

Vision Language Models Cannot Reason About Physical Transformation

arXiv 2603.07109 ↗First posted 7 Mar 2026cs.AIcited by 4

On the map: LLM reasoning → Spatial and embodied reasoning

also in 1 more
Abstract (arXiv)

Understanding physical transformations is fundamental for reasoning in dynamic environments. While Vision Language Models (VLMs) show promise in embodied applications, whether they genuinely understand physical transformations remains unclear. We introduce ConservationBench evaluating conservation -- whether physical quantities remain invariant under transformations. Spanning four properties with paired conserving/non-conserving scenarios, we generate and evaluate 23,040 questions across 112 VLMs. Results reveal systematic failure: performance remains near chance with improvements on conservation tasks accompanied by drops on controls. Control experiments show strong textual priors favoring invariance, yet models perform worse with actual visual content when performance is balanced across conserving and non-conserving scenarios. Neither temporal resolution, prompting, nor curated sampling helps. These findings show that current VLMs fail to maintain transformation-invariant representations of physical properties across dynamic scenes.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 47 of its 106 references that are in the map. First posted on arXiv 7 Mar 2026.

Builds on

47 of 106 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025PhysUniBench: A Multi-Modal Physics Reasoning Benchmark at Undergraduate Level21 Jun
  2. 2025PHYBench: Holistic Evaluation of Physical Perception and Reasoning in Large Language Models22 Apr
  3. 2025Cosmos-Reason1: From Physical Common Sense To Embodied Reasoning18 Mar
  4. 2024Probing Mechanical Reasoning in Large Vision Language Models1 Oct
  5. 2024PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMs12 Feb
  6. 2024ContPhy: Continuum Physical Concept Learning and Reasoning from Videos9 Feb

Other references · 41

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Scaling Laws for Neural Language Modelscited by 6,411
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  4. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  5. 2022Emergent Abilities of Large Language Modelscited by 2,483
  6. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  7. 2025Gemma 3 Technical Reportcited by 1,686
  8. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  9. 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 749
  10. 2022When and why vision-language models behave like bags-of-words, and what to do about it?cited by 603
  11. 2018From Recognition to Cognition: Visual Commonsense Reasoningcited by 553
  12. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 550
29 more
  1. 2020Scaling Laws for Autoregressive Generative Modelingcited by 510
  2. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 466
  3. 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Stepcited by 446
  4. 2024Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modelingcited by 209
  5. 2024Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generationcited by 167
  6. 2023Physically Grounded Vision-Language Models for Robotic Manipulationcited by 163
  7. 2025Do generative video models understand physical principles?cited by 151
  8. 2025PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understandingcited by 147
  9. 2021Physion: Evaluating Physical Prediction from Vision in Humans and Machinescited by 145
  10. 2024Vision language models are blind: Failing to translate detailed visual features into wordscited by 144
  11. 2024Improve Vision Language Model Chain-of-thought Reasoningcited by 135
  12. 2024Vision-Language Models Can Self-Improve Reasoning via Reflectioncited by 66
  13. 2025VLIPP: Towards Physically Plausible Video Generation with Vision and Language Informed Physical Priorcited by 49
  14. 2024Core Knowledge Deficits in Multi-Modal Language Modelscited by 38
  15. 2025Emergent Abilities in Large Language Models: A Surveycited by 34
  16. 2025Holistic Evaluation of Multimodal LLMs on Spatial Intelligencecited by 28
  17. 2024Exploring Perceptual Limitation of Multimodal Large Language Modelscited by 24
  18. 2025Generative Physical AI in Vision: A Surveycited by 24
  19. 2025Do Vision-Language Models Have Internal World Models? Towards an Atomic Evaluationcited by 23
  20. 2022CRIPP-VQA: Counterfactual Reasoning about Implicit Physical Properties via Video Question Answeringcited by 13
  21. 2024Vision Language Models See What You Want but not What You Seecited by 10
  22. 2025Hidden in plain sight: VLMs overlook their visual representationscited by 10
  23. 2025VIDEOP2R: Video Understanding from Perception to Reasoningcited by 8
  24. 2024Do Pre-trained Vision-Language Models Encode Object States?cited by 7
  25. 2025Probing Perceptual Constancy in Large Vision-Language Modelscited by 7
  26. 2025The Philosophical Foundations of Growing AI Like A Childcited by 6
  27. 2020VisualCOMET: Reasoning about the Dynamic Context of a Still Imagecited by 5
  28. 2025Testing the Limits of Fine-Tuning for Improving Visual Cognition in Vision Language Modelscited by 4
  29. 2025Rethinking the Simulation vs. Rendering Dichotomy: No Free Lunch in Spatial World Modellingcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Code as Worlds: Agentic Discovery of Executable World Representations for Physical Reasoningcited by 6
  2. 2026World Models in Words: Auditing Physical State-Transition Commitments in Vision-Language Modelscited by 0
  3. 2026Physics as the label for measuring and correcting materials reasoning in multimodal modelscited by 0
  4. 2026Training Object Permanence in World Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Semantic Richness or Geometric Reasoning? The Fragility of VLM's Visual Invariance
  2. 2023Can Language Models Understand Physical Concepts?
  3. 2026Vision-Language-Action in Robotics: A Survey of Datasets, Benchmarks, and Data Engines
  4. 2026From Perception to Action: An Interactive Benchmark for Vision Reasoning
  5. 2026PhysVista: Benchmarking Physical Intelligence in VLMs via a Perception-Reasoning-Assessment Loop
  6. 2026Unmasking the Illusion of Embodied Reasoning in Vision-Language-Action Models
  7. 2026Where To Look? : Causal Tracing of Vision Encoders in VLM
  8. 2025Fake or Real, Can Robots Tell? Evaluating VLM Robustness to Domain Shift in Single-View Robotic Scene Understanding