mlsanityData to 7 Oct 2026

PhysAlign: A Benchmark for Evidence-Grounded Role Alignment in Multimodal Physics Reasoning

arXiv 2609.33319 ↗First posted 27 Sep 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

A key challenge in physics diagram understanding is correctly associating visual information with the physical entities, relations, and conditions it describes. Even when a value, symbol, or other local element is accurately recognized, assigning it to the wrong entity or scope can distort the underlying physical premise and lead to incorrect reasoning. To systematically study this challenge, we introduce PhysAlign, a benchmark designed to assess whether multimodal models correctly associate information recognized from physics diagrams with its intended physical role. By disentangling visual recognition from physical-role assignment through localized probes and controlled variants, PhysAlign isolates correspondence errors from recognition failures. It contains 3,341 human-validated probes spanning 986 physics problems, enabling systematic evaluation of visual recognition and physical-role correspondence at scale. We further introduce five complementary evaluation metrics, including CAcc, GAcc, and JAcc, which provide a comprehensive assessment of models' ability to recognize diagram content, establish correct physical correspondences, and solve the underlying physics problem. Across our evaluated multimodal models, PhysAlign reveals a consistent gap between local visual recognition and physical-role grounding. Even when the queried content is correctly recognized, the conditional correspondence error rate remains 13.8% for GPT-6-Astra and rises to about 50.6% for InternVL3.5-8B. These findings indicate that strong perception alone does not ensure reliable physical interpretation, exposing a distinct grounding bottleneck that is largely hidden by answer-level accuracy and highlighting the need for future models to better align recognized visual evidence with its physical meaning.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 23 of its 29 references that are in the map. First posted on arXiv 27 Sep 2026.

Builds on

23 of 29 references.

References · 23

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  3. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  4. 2019Towards VQA Models That Can Readcited by 1,696
  5. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  6. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  7. 2018Datasheets for Datasetscited by 1,371
  8. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  9. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 893
  10. 2022When and why vision-language models behave like bags-of-words, and what to do about it?cited by 603
  11. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 573
  12. 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 572
11 more
  1. 2022Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionalitycited by 523
  2. 2023SugarCrepe: Fixing Hackable Benchmarks for Vision-Language Compositionalitycited by 238
  3. 2022VL-CheckList: Evaluating Pre-trained Vision-Language Models with Objects, Attributes and Relationscited by 113
  4. 2019AI2D-RST: A multimodal corpus of 1000 primary school science diagramscited by 60
  5. 2025PhysReason: A Comprehensive Benchmark towards Physics-Based Reasoningcited by 58
  6. 2025PhyX: Does Your Model Have the "Wits" for Physical Reasoning?cited by 37
  7. 2025SeePhys: Does Seeing Help Thinking? -- Benchmarking Vision-Based Physics Reasoningcited by 32
  8. 2025PhysicsArena: The First Multimodal Physics Reasoning Benchmark Exploring Variable, Process, and Solution Dimensionscited by 8
  9. 2026LiveK12Bench: Have Large Multimodal Models Truly Conquered High School-level Examinations?cited by 3
  10. 2026SeePhys Pro: Diagnosing Modality Transfer and Blind-Training Effects in Multimodal RLVR for Physics Reasoningcited by 1
  11. 2026PhysElite: How Far Are LLMs from Solving Olympiad-Level Physics Problems?cited by 1

Built on by

Not cited yet.