mlsanityData to 14 Oct 2026

Physics as the label for measuring and correcting materials reasoning in multimodal models

arXiv 2609.12181 ↗First posted 10 Sep 2026cs.CV cond-mat.mtrl-scicited by 0

On the map: LLM reasoning → Clinical and biomedical applications

Abstract (arXiv)

Vision-language and language models increasingly interpret materials data, yet benchmarks report that they hallucinate invalid properties and violate physical law. Evaluation matches final answers to scarce human labels, while discovery agents verify final proposals or density functional theory (DFT) execution. Neither measures the physical consistency of a model's reasoning chain. Materials data carries its own physics, making a large class of materials reasoning verifiable without annotation. We introduce MatPCR, a label-free benchmark whose programmatic oracles check diffraction geometry through Bragg's law, scale bars, spectral peaks, and Materials Project-grounded checks of near-hull stability, computed band-gap class, and net magnetization. We define the Physical-Consistency Rate over image and structure inputs; introduce Constraint-Grounded Self-Verification, an agentic loop whose gain survives self-refinement and equal-compute re-prompting controls; release an open verifier useful in distribution but near chance on all six held-out constraint types; and derive an exact identity for how oracle error displaces the reported rate.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 37 of its 57 references that are in the map. First posted on arXiv 10 Sep 2026.

Builds on

37 of 57 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026Toward Guarantees for Clinical Reasoning in Vision Language Models via Formal Verification27 Feb
  2. 2025AtomWorld: A Benchmark for Evaluating Spatial Reasoning in Large Language Models on Crystalline Materials6 Oct
  3. 2025Aligning Reasoning LLMs for Materials Discovery with Physics-aware Rejection Sampling31 Aug

Other references · 34

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  4. 2024ProcessBench: Identifying Process Errors in Mathematical Reasoningcited by 292
  5. 2024HoneyComb: A Flexible LLM-Based Agent System for Materials Sciencecited by 57
  6. 2025VLM-R3: Region Recognition, Reasoning, and Refinement for Enhanced Multimodal Chain-of-Thoughtcited by 49
  7. 2024LLMatDesign: Autonomous Materials Discovery with Large Language Modelscited by 42
  8. 2025CompassVerifier: A Unified and Robust Verifier for LLMs Evaluation and Outcome Rewardcited by 40
  9. 2025MM-Verify: Enhancing Multimodal Reasoning with Chain-of-Thought Verificationcited by 39
  10. 2024AtomAgents: Alloy design and discovery through physics-aware multi-modal multi-agent artificial intelligencecited by 36
  11. 2025DREAMS: Density Functional Theory Based Research Engine for Agentic Materials Simulationcited by 29
  12. 2020Overview: Computer vision and machine learning for microstructural characterization and analysiscited by 22
22 more
  1. 2024LLM4Mat-Bench: Benchmarking Large Language Models for Materials Property Predictioncited by 21
  2. 2025MatterChat: A Multi-Modal LLM for Material Sciencecited by 18
  3. 2025HalluVerse25: Fine-grained Multilingual Benchmark Dataset for LLM Hallucinationscited by 14
  4. 2025T1: Tool-integrated Verification for Test-time Compute Scaling in Small Language Modelscited by 13
  5. 2026Beyond Outcome Verification: Verifiable Process Reward Models for Structured Reasoningcited by 13
  6. 2025Toward Greater Autonomy in Materials Discovery Agents: Unifying Planning, Physics, and Scientistscited by 10
  7. 2025Establishing baselines for generative discovery of inorganic crystalscited by 9
  8. 2025Can Multimodal LLMs See Materials Clearly? A Multimodal Benchmark on Materials Characterizationcited by 9
  9. 2025A Survey of AI for Materials Science: Foundation Models, LLM Agents, Datasets, and Toolscited by 7
  10. 2026P1-VL: Bridging Visual Perception and Scientific Reasoning in Physics Olympiadscited by 7
  11. 2024Trust but Verify: Programmatic VLM Evaluation in the Wildcited by 5
  12. 2025CoRGI: Verified Chain-of-Thought Reasoning with Post-hoc Visual Groundingcited by 4
  13. 2026Improving Data and Reward Design for Scientific Reasoning in Large Language Modelscited by 4
  14. 2026Vision Language Models Cannot Reason About Physical Transformationcited by 4
  15. 2026IRIS: A Real-World Benchmark for Inverse Recovery and Identification of Physical Dynamic Systems from Monocular Videocited by 4
  16. 2025opXRD: Open Experimental Powder X-ray Diffraction Databasecited by 3
  17. 2026Self-Verification Dilemma: Experience-Driven Suppression of Overused Checking in LLM Reasoningcited by 3
  18. 2026Faithfulness as Information Flow: Evaluating and Training Faithful Chain-of-Thought Reasoningcited by 3
  19. 2026CrystalXRD-Bench: Benchmarking Vision-Language Models for XRD Peak Indexing Across Diverse Crystalline Materialscited by 2
  20. 2026SCI-PRM: A Tool Aware Process Reward Model for Scientific Reasoning Verificationcited by 2
  21. 2026How Far Can You Grow? Characterizing the Extrapolation Frontier of Graph Generative Models for Materials Sciencecited by 1
  22. 2026EAA: Automating materials characterization with vision language model agentscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026SCALAR: Quantifying Structural Hallucination, Consistency, and Reasoning Gaps in Materials Foundation Models
  2. 2026MATRIX: A Multimodal Benchmark and Post-Training Framework for Materials Science
  3. 2026OmniMatBench: A Human-Calibrated Multimodal Reasoning Benchmark Across 19 Materials Science Subfields
  4. 2026MatPhaseBench: A Semantics-Guided Benchmark for Materials Phase Diagrams Understanding
  5. 2026ARIA: A Causal-Aware Framework for Rescuing LLM Reasoning in Trustworthy Materials Discovery
  6. 2026MaterialFigBENCH: benchmark dataset with figures for evaluating college-level materials science problem-solving abilities of multimodal large language models
  7. 2026From Data to Theory: Autonomous Large Language Model Agents for Materials Science
  8. 2025MatSciBench: Benchmarking the Reasoning Ability of Large Language Models in Materials Science