mlsanityData to 14 Oct 2026

Grading the Unspoken: Evaluating Tacit Reasoning in Quantum Field Theory and String Theory with LLMs

arXiv 2604.14188 ↗First posted 1 Apr 2026cs.AI cs.CL hep-thcited by 1

On the map: LLM reasoning → Spatial and embodied reasoning

Abstract (arXiv)

Large language models have demonstrated impressive performance across many domains of mathematics and physics. One natural question is whether such models can support research in highly abstract theoretical fields such as quantum field theory and string theory. Evaluating this possibility faces an immediate challenge: correctness in these domains is layered, tacit, and fundamentally non-binary. Standard answer-matching metrics fail to capture whether intermediate conceptual steps are properly reconstructed or whether implicit structural constraints are respected. We construct a compact expert-curated dataset of twelve questions spanning core areas of quantum field theory and string theory, and introduce a five-level grading rubric separating statement correctness, key concept awareness, reasoning chain presence, tacit step reconstruction, and enrichment. Evaluating multiple contemporary LLMs, we observe near-ceiling performance on explicit derivations within stable conceptual frames, but systematic degradation when tasks require reconstruction of omitted reasoning steps or reorganization of representations under global consistency constraints. These failures are driven not only by missing intermediate steps, but by an instability in representation selection: models often fail to identify the correct conceptual framing required to resolve implicit tensions. We argue that highly abstract theoretical physics provides a uniquely sensitive lens on the epistemic limits of current evaluation paradigms.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 40 of its 48 references that are in the map. First posted on arXiv 1 Apr 2026.

Builds on

40 of 48 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025ABench-Physics: Benchmarking Physical Reasoning in LLMs via High-Difficulty and Dynamic Physics Problems7 Jul
  2. 2025Scaling Physical Reasoning with the PHYSICS Dataset21 May
  3. 2025PHYBench: Holistic Evaluation of Physical Perception and Reasoning in Large Language Models22 Apr
  4. 2025Theoretical Physics Benchmark (TPBench) -- a Dataset and Study of AI Reasoning Capabilities in Theoretical Physics19 Feb
  5. 2025PhysReason: A Comprehensive Benchmark towards Physics-Based Reasoning17 Feb
  6. 2025UGPhysics: A Comprehensive Benchmark for Undergraduate Physics Reasoning with Large Language Models1 Feb

Other references · 34

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  8. 2023Let's Verify Step by Stepcited by 3,789
  9. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  10. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  11. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  12. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
22 more
  1. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,076
  2. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 560
  3. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 558
  4. 2022Large Language Models are Better Reasoners with Self-Verificationcited by 307
  5. 2023Faithful Chain-of-Thought Reasoningcited by 288
  6. 2024Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Modelscited by 284
  7. 2023Mathematical Capabilities of ChatGPTcited by 268
  8. 2023SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Modelscited by 256
  9. 2024FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AIcited by 243
  10. 2023Universal Self-Consistency for Large Language Model Generationcited by 203
  11. 2022COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamicscited by 161
  12. 2024Multi-Step Reasoning with Large Language Models, a Surveycited by 143
  13. 2023Algorithm of Thoughts: Enhancing Exploration of Ideas in Large Language Modelscited by 92
  14. 2023Evaluating Language Models for Mathematics through Interactionscited by 70
  15. 2024Evaluating Mathematical Reasoning Beyond Accuracycited by 69
  16. 2023ARB: Advanced Reasoning Benchmark for Large Language Modelscited by 60
  17. 2025CRANE: Reasoning with constrained LLM generationcited by 38
  18. 2024Quantum Many-Body Physics Calculations with Large Language Modelscited by 17
  19. 2024Aligning with Logic: Measuring, Evaluating and Improving Logical Preference Consistency in Large Language Modelscited by 12
  20. 2025Learning-at-Criticality in Large Language Models for Quantum Field Theory and Beyondcited by 5
  21. 2025Bayesian Network Structure Discovery Using Large Language Modelscited by 5
  22. 2025FeynTune: Large Language Models for High-Energy Theorycited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Fine-Tuning Small Reasoning Models for Quantum Field Theory
  2. 2024Exploring the Limits of Fine-grained LLM-based Physics Inference via Premise Removal Interventions
  3. 2026Scientific Logicality Enriched Methodology for LLM Reasoning: A Practice in Physics
  4. 2025Atomic Thinking of LLMs: Decoupling and Exploring Mathematical Reasoning Abilities
  5. 2026FeynmanBench: Benchmarking Multimodal LLMs on Diagrammatic Physics Reasoning
  6. 2025Semantic Wave Functions: Exploring Meaning in Large Language Models Through Quantum Formalism
  7. 2026Quantum-Audit: Evaluating the Reasoning Limits of LLMs on Quantum Computing
  8. 2025Can Theoretical Physics Research Benefit from Language Agents?