MechReasoner: A Simulator and Benchmark for Mechanistic Reasoning in Qualitative Physics
On the map: LLM reasoning → Reasoning benchmarks and evaluation also fits: Test-time compute and efficient thinking
Abstract (arXiv)
This work introduces MechReasoner, a mechanistic qualitative simulator grounded in confluence-based qualitative physics, together with a benchmark for mechanistic inference. Current large language models (LLMs) generate fluent mechanistic descriptions that do not reliably follow from underlying structural and causal constraints. The benchmark tests whether answers preserve simulator-licensed ambiguity, quantified claims, episode-graph transition evidence, repairs, and trace-support judgments. Its 1,120 items are generated deterministically from admissible interpretation sets, component states, scenario restrictions, confluence constraints, and derivation steps across 18 catalog mechanisms and six task families. Each mechanism undergoes converter checks of structure and topology and behavioral checks against quantitative simulations. GPT-5.5 accuracy decreases as family-specific mechanistic complexity increases, from 76.1% in the lowest-complexity bucket (B1) to 38.0% in the highest-complexity bucket (B4). The negative association remains after controls for rendered-prompt and expected-answer length. These results show that qualitative simulators can support auditable NLP benchmarks for mechanistic inference.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 1 of its 28 references that are in the map. First posted on arXiv 28 Sep 2026.
Builds on
1 of 28 references.
Closest ancestors · 1
Same direction, newest first
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026PhysMent: An Interactive Approach For LLM Reasoning In Physics Problems
- 2023Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models
- 2025A Statistical Physics of Language Model Reasoning
- 2026Reason, Reward, Refine: Step-Level Errors Corrections with Structured Feedback for Physics Reasoning in Small Language Models
- 2026Causally Grounded Mechanistic Interpretability for LLMs with Faithful Natural-Language Explanations
- 2026Integrating Simulation and Chain-of-thought Reasoning in Multimodal-Language Models For Physical Reasoning
- 2025PhysicsArena: The First Multimodal Physics Reasoning Benchmark Exploring Variable, Process, and Solution Dimensions
- 2025Atom of Thoughts for Markov LLM Test-Time Scaling