mlsanityData to 7 Oct 2026

STEMVerse: A Dual-Axis Diagnostic Framework for STEM Reasoning in Large Language Models

arXiv 2602.02497 ↗First posted 14 Jan 2026cs.CL cs.AIcited by 1

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

As Large Language Models (LLMs) achieve significant breakthroughs in complex reasoning tasks, evaluating their proficiency in science, technology, engineering, and mathematics (STEM) has become a primary method for measuring machine intelligence. However, current evaluation paradigms often treat benchmarks as isolated "silos," offering only monolithic aggregate scores that neglect the intricacies of both academic specialization and cognitive depth. This result-oriented approach fails to distinguish whether model errors stem from insufficient domain knowledge or deficiencies in cognitive capacity, thereby limiting the diagnostic value. To address this, we propose STEMVerse, a diagnostic framework designed to systematically analyze the STEM reasoning capabilities of LLMs. This framework characterizes model performance across academic specialization and cognitive complexity to map the capability required for reasoning. We re-aggregate over 20,000 STEM problems from mainstream benchmarks into a unified "Discipline × Cognition" capability space, assigning dual-axis labels to every instance. Utilizing this unified diagnostic framework, we systematically evaluate representative LLM families across varying parameter scales and training paradigms. Our empirical results reveal structural failure patterns in STEM reasoning. By integrating multi-disciplinary coverage and fine-grained cognitive stratification into a unified framework, STEMVerse provides a clear and actionable perspective for understanding the scientific reasoning characteristics of LLMs.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 30 of its 49 references that are in the map. First posted on arXiv 14 Jan 2026.

Builds on

30 of 49 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking8 Jan
  2. 2024Large Language Models for Mathematical Reasoning: Progresses and Challenges31 Jan
  3. 2023SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models20 Jul

Other references · 27

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  5. 2024Qwen2.5 Technical Reportcited by 4,713
  6. 2024GPT-4o System Cardcited by 4,467
  7. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  8. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 810
  9. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  10. 2025Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learningcited by 260
  11. 2024SciAgent: Tool-augmented Language Models for Scientific Reasoningcited by 69
  12. 2025SuperGPQA: Scaling LLM Evaluation across 285 Graduate Disciplinescited by 67
15 more
  1. 2025MegaScience: Pushing the Frontiers of Post-Training Datasets for Science Reasoningcited by 56
  2. 2025Large Language Models for Planning: A Comprehensive and Systematic Surveycited by 38
  3. 2024Aviary: training language agents on challenging scientific taskscited by 23
  4. 2024Improving Physics Reasoning in Large Language Models Using Mixture of Refinement Agentscited by 22
  5. 2025VerifyBench: A Systematic Benchmark for Evaluating Reasoning Verifiers Across Domainscited by 22
  6. 2025Chemical reasoning in LLMs unlocks strategy-aware synthesis planning and reaction mechanism elucidationcited by 17
  7. 2025Review of Case-Based Reasoning for LLM Agents: Theoretical Foundations, Architectural Components, and Cognitive Integrationcited by 12
  8. 2025Do Cognitively Interpretable Reasoning Traces Improve LLM Performance?cited by 11
  9. 2025CausalStep: A Benchmark for Explicit Stepwise Causal Reasoning in Videoscited by 9
  10. 2025Look Less, Reason More: Rollout-Guided Adaptive Pixel-Space Reasoningcited by 9
  11. 202511Plus-Bench: Demystifying Multimodal LLM Spatial Reasoning with Cognitive-Inspired Analysiscited by 8
  12. 2025SciAgent: A Unified Multi-Agent System for Generalistic Scientific Reasoningcited by 6
  13. 2025ATLAS: A High-Difficulty, Multidisciplinary Benchmark for Frontier Scientific Reasoningcited by 5
  14. 2025Cognitive Mirrors: Exploring the Diverse Functional Roles of Attention Heads in LLM Reasoningcited by 5
  15. 2025Select Less, Reason More: Prioritizing Evidence Purity for Video Reasoningcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Large Language Models and their Awareness of Mechanics and Spatial Geometrycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Logics-STEM: Empowering LLM Reasoning via Failure-Driven Post-Training and Document Knowledge Enhancement
  2. 2025STEM: Efficient Relative Capability Evaluation of LLMs through Structured Transition Samples
  3. 2025EngiBench: A Benchmark for Evaluating Large Language Models on Engineering Problem Solving
  4. 2026General365: Benchmarking General Reasoning in Large Language Models Across Diverse and Challenging Tasks
  5. 2025Large Language Models and Mathematical Reasoning Failures
  6. 2023Have LLMs Advanced Enough? A Challenging Problem Solving Benchmark For Large Language Models
  7. 2025Evaluating Mathematical Reasoning Across Large Language Models: A Fine-Grained Approach
  8. 2026ReEfBench: Quantifying the Reasoning Efficiency of LLMs