mlsanityData to 14 Oct 2026

SCALAR: Quantifying Structural Hallucination, Consistency, and Reasoning Gaps in Materials Foundation Models

arXiv 2601.22312 ↗First posted 29 Jan 2026cs.LG cond-mat.mtrl-sci cs.CEcited by 0

On the map: LLM reasoning → Clinical and biomedical applications

Abstract (arXiv)

Large language models are increasingly applied to materials science reasoning, yet their behavior under physically structured distribution shifts remains poorly understood. We introduce SCALAR (Structural Consistency And Logic Across Regimes), a benchmark for evaluating geometric scale generalization and its connection to structural hallucination, consistency, and reasoning in materials foundation models. Given canonical crystal representations, models must reason about derived nanoparticle structures obtained through supercell expansion and geometric truncation across length scales spanning a few atoms to over 18,000 atoms, totaling ≈100,000 structures from DFT-validated unit cells. SCALAR defines three tasks. (i) CIF to property prediction. (ii) A Chain-of-Thought variant with explicit physics-grounded reasoning. (iii) Inverse retrieval identifying crystals from candidates given target properties. Outputs are evaluated via structured metrics capturing numeric error, hallucination, cross-prompt consistency, monotonic reasoning, output validity, and retrieval regret. Experiments across diverse foundation models reveal large, model-dependent shifts under explicit reasoning, often reducing hallucination and error, but frequently destabilizing consistency or validity. These results demonstrate that geometric scale generalization cannot be inferred from accuracy alone. Supplementary materials are available at https://github.com/KurbanIntelligenceLab/SCALAR.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 15 of its 39 references that are in the map. First posted on arXiv 29 Jan 2026.

Builds on

15 of 39 references.

References · 15

  1. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  2. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 900
  3. 2021E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentialscited by 708
  4. 2023ChemCrow: Augmenting large-language models with chemistry toolscited by 631
  5. 2020Directional Message Passing for Molecular Graphscited by 561
  6. 2024A Review of Large Language Models and Autonomous Agents in Chemistrycited by 99
  7. 202314 Examples of How LLMs Can Transform Materials Science and Chemistry: A Reflection on a Large Language Model Hackathoncited by 86
  8. 2024FlowLLM: Flow Matching for Material Generation with Large Language Models as Base Distributionscited by 53
  9. 2023Multimodal Foundation Models for Material Property Prediction and Discoverycited by 19
  10. 2024Foundational Large Language Models for Materials Researchcited by 18
  11. 2023Equivariant Message Passing Neural Network for Crystal Material Discoverycited by 10
  12. 2024DARWIN 1.5: Large Language Models as Materials Science Adapted Learnerscited by 9
3 more
  1. 2025MatterTune: An Integrated, User-Friendly Platform for Fine-Tuning Atomistic Foundation Models to Accelerate Materials Simulation and Discoverycited by 5
  2. 2025Stress-Testing Multimodal Foundation Models for Crystallographic Reasoningcited by 4
  3. 2025Beyond Atomic Geometry Representations in Materials Science: A Human-in-the-Loop Multimodal Frameworkcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Physics as the label for measuring and correcting materials reasoning in multimodal models
  2. 2025AtomWorld: A Benchmark for Evaluating Spatial Reasoning in Large Language Models on Crystalline Materials
  3. 2026MatMind: A Structure-Activity Knowledge-Driven Generative Foundation Model for Materials Science
  4. 2026ARIA: A Causal-Aware Framework for Rescuing LLM Reasoning in Trustworthy Materials Discovery
  5. 2026CrystalReasoner: Reasoning and RL for Property-Conditioned Crystal Structure Generation
  6. 2024LLaMP: Large Language Model Made Powerful for High-fidelity Materials Knowledge Retrieval and Distillation
  7. 2026From Data to Theory: Autonomous Large Language Model Agents for Materials Science
  8. 2025LacMaterial: Large Language Models as Analogical Chemists for Materials Discovery