mlsanityData to 7 Oct 2026

NICE FACT: Diagnosing and Calibrating VLMs in Quantitative Reasoning for Kinematic Physics

arXiv 2605.08452 ↗First posted 8 May 2026cs.CVcited by 0

On the map: LLM reasoning → Spatial and embodied reasoning

also in 1 more
Abstract (arXiv)

The ability to derive precise spatial and physical insights is a cornerstone of vision-language models (VLMs), yet their poor performances in related spatial intelligence tasks such as physical reasoning remain a fundamental barrier. The community critically lacks a scientific analysis revealing whether VLMs faithfully reach answers or plausibly make guesses. This work aims to provide a fundamental understanding of how VLMs perceive the physical world, and utilize physical laws, while assessing the reliability of model confidence. We propose NICE and FACT, a dual-diagnostic paradigm that explicitly decomposes quantitative reasoning for kinematic physics: FACT diagnoses visual fidelity, physical law comprehension, and temporal grounding. NICE studies our novel neighborhood-informed calibration method and novel metrics to evaluate and calibrate confidence reliability. Evaluated across 6 latest state-of-the-art VLMs, we uncover that models fail to identify visual preconditions or utilize necessary physical laws to reach answers. This work highlights and establishes a standardized diagnostic paradigm to guide the development of faithful, physically-grounded VLMs.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 26 of its 33 references that are in the map. First posted on arXiv 8 May 2026.

Builds on

26 of 33 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025From Indoor to Open World: Revealing the Spatial Reasoning Gap in MLLMs22 Dec
  2. 2025QuantiPhy: A Quantitative Benchmark Evaluating Physical Reasoning Abilities of Vision-Language Models22 Dec
  3. 2025Reasoning in Space via Grounding in the World15 Oct
  4. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces18 Dec
  5. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities22 Jan

Other references · 21

  1. 2017On Calibration of Modern Neural Networkscited by 4,860
  2. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  3. 2025Qwen3-VL Technical Reportcited by 2,534
  4. 2015VQA: Visual Question Answeringcited by 2,489
  5. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  6. 2024Phi-4 Technical Reportcited by 675
  7. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 571
  8. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 550
  9. 2025Qwen3-Omni Technical Reportcited by 451
  10. 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 439
  11. 2025SmolVLM: Redefining small and efficient multimodal modelscited by 226
  12. 2024STAR: A Benchmark for Situated Reasoning in Real-World Videoscited by 213
9 more
  1. 2025PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understandingcited by 147
  2. 2026Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Groundingcited by 132
  3. 2025Cambrian-S: Towards Spatial Supersensing in Videocited by 123
  4. 2022Stop Measuring Calibration When Humans Disagreecited by 70
  5. 2024Evaluating Multiview Object Consistency in Humans and Image Modelscited by 12
  6. 2024Mind the Uncertainty in Human Disagreement: Evaluating Discrepancies between Model Predictions and Human Responses in VQAcited by 6
  7. 2025Unveiling the "Fairness Seesaw": Discovering and Mitigating Gender and Race Bias in Vision-Language Modelscited by 4
  8. 2026From Generative Engines to Actionable Simulators: The Imperative of Physical Grounding in World Modelscited by 4
  9. 2025Human Uncertainty-Aware Data Selection and Automatic Labeling in Visual Question Answeringcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Physics Knowledge in Frontier Models: A Diagnostic Study of Failure Modes
  2. 2025Interpretable Physics Reasoning and Performance Taxonomy in Vision-Language Models
  3. 2024Physics Context Builders: A Modular Framework for Physical Reasoning in Vision-Language Models
  4. 2026PhysVista: Benchmarking Physical Intelligence in VLMs via a Perception-Reasoning-Assessment Loop
  5. 2026PhysicsMind: Sim and Real Mechanics Benchmarking for Physical Reasoning and Prediction in Foundational VLMs and World Models
  6. 2025From Diagnosis to Improvement: Probing Spatio-Physical Reasoning in Vision Language Models
  7. 2026Answer-Level Trust Selection for Physical Vision-Language Reasoning
  8. 2026P1-VL: Bridging Visual Perception and Scientific Reasoning in Physics Olympiads