mlsanityData to 7 Oct 2026

The Periodic Table of LLM Reasoning: A Structured Survey of Reasoning Paradigms, Methods, and Failure Modes

arXiv 2606.11470 ↗First posted 9 Jun 2026cs.CLcited by 0

On the map: LLM reasoning → Reasoning benchmarks and evaluation

also in 1 more
Abstract (arXiv)

Reasoning has become central to how Large Language Models (LLMs) are evaluated and interpreted, spanning Chain-of-Thought (CoT), mathematical problem-solving, multi-hop question answering, code generation, retrieval-augmented reasoning, tool use, and multimodal decision-making. In this survey, we introduce the Periodic Table of LLM Reasoning, a framework organizing 300+ recent papers by reasoning paradigm, methodological mechanism, evaluation setting, and failure mode. We classify LLM reasoning into nine paradigms: Chain-of-Thought, Multi-Hop, Mathematical, Commonsense, Visual and Temporal, Code and Algorithmic, Retrieval-Augmented, Tool-Augmented or Agentic, and Reinforcement Learning-based reasoning. For each, we review approaches, including prompting, architectural interventions, supervised fine-tuning, verifier-guided inference, reward modeling, retrieval, tool interfaces, agentic workflows, and benchmark design. We argue that LLM reasoning is not a single emergent capability but a family of scaffolded behaviors shaped by model scale, task structure, external memory, supervision, and evaluation protocols. We synthesize recurring failure modes, including hallucinated reasoning, brittle multi-step inference, spurious rationales, weak causal grounding, poor out-of-distribution generalization, benchmark contamination, and unreliable self-verification. Progress is difficult to compare across paradigms because gains may arise from prompting, retrieval, verifier design, or benchmark structure rather than general reasoning ability. The survey connects methods to their assumptions, strengths, and failure modes, providing a reference map of the field and a diagnostic framework for future work. We conclude that robust LLM reasoning will require meta-reasoning, multimodal and temporal grounding, adaptive tool use, and principled evaluation under distribution shift.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 262 of its 342 references that are in the map. First posted on arXiv 9 Jun 2026.

Builds on

262 of 342 references.

Closest ancestors · 36

Same direction, newest first

  1. 2025mSCoRe: a Multilingual and Scalable Benchmark for Skill-based Commonsense Reasoning13 Aug
  2. 2025GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models8 Aug
  3. 2025MultiNRC: A Challenging and Native Multilingual Reasoning Evaluation Benchmark for LLMs23 Jul
  4. 2025Who Benchmarks the Benchmarks? Towards Comprehensive Evaluation of Commonsense Reasoning Benchmarks10 Apr
  5. 2025DeepSeek-R1 Thoughtology: Let's think about LLM Reasoning2 Apr
  6. 2025JudgeLRM: Large Reasoning Models as a Judge31 Mar
  7. 2025Landscape of Thoughts: Visualizing the Reasoning Process of Large Language Models28 Mar
  8. 2025I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoders24 Mar
  9. 2025Towards Better Understanding of Program-of-Thought Reasoning in Cross-Lingual and Multilingual Environments25 Feb
  10. 2025Does Reasoning Introduce Bias? A Study of Social Bias Evaluation and Mitigation in LLM Reasoning21 Feb
  11. 2025A Survey of Multilingual Reasoning in Language Models13 Feb
  12. 2025Adapting Language-Specific LLMs to a Reasoning Model in One Day via Model Merging -- An Open Recipe13 Feb
24 more
  1. 2025Great Models Think Alike and this Undermines AI Oversight6 Feb
  2. 2025ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoning3 Feb
  3. 2025Large Language Models Think Too Fast To Explore Effectively29 Jan
  4. 2025Reasoning Language Models: A Blueprint20 Jan
  5. 2025Multiple Choice Questions: Reasoning Makes Large Language Models (LLMs) More Self-Confident, Especially When They are Wrong16 Jan
  6. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Models16 Jan
  7. 2025SLAM: Towards Efficient Multilingual Reasoning via Selective Language Alignment7 Jan
  8. 2024LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks19 Dec
  9. 2024Are Your LLMs Capable of Stable Reasoning?17 Dec
  10. 2024SocialGPT: Prompting LLMs for Social Relation Reasoning via Greedy Segment Optimization28 Oct
  11. 2024Iteration of Thought: Leveraging Inner Dialogue for Autonomous Large Language Model Reasoning19 Sep
  12. 2024Large Language Models Are Cross-Lingual Knowledge-Free Reasoners24 Jun
  13. 2024BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack14 Jun
  14. 2024MindMerger: Efficient Boosting LLM Reasoning in non-English Languages27 May
  15. 2024Advancing LLM Reasoning Generalists with Preference Trees2 Apr
  16. 2024Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Models19 Feb
  17. 2024Understanding Reasoning Ability of Language Models From the Perspective of Reasoning Paths Aggregation5 Feb
  18. 2024Efficient Tool Use with Chain-of-Abstraction Reasoning30 Jan
  19. 2024LangBridge: Multilingual Reasoning Without Multilingual Supervision19 Jan
  20. 2024MAPO: Advancing Multilingual Reasoning through Multilingual Alignment-as-Preference Optimization12 Jan
  21. 2023Understanding Social Reasoning in Language Models with Language Models21 Jun
  22. 2023Large Language Models Are Not Strong Abstract Reasoners31 May
  23. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity8 Feb
  24. 2022ReAct: Synergizing Reasoning and Acting in Language Models6 Oct

Other references · 226

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  6. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  7. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  8. 2022Emergent Abilities of Large Language Modelscited by 2,483
  9. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  10. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  11. 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 740
  12. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 573
38 more
  1. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMscited by 564
  2. 2024GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Modelscited by 533
  3. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 527
  4. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 518
  5. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 501
  6. 2025LIMO: Less is More for Reasoningcited by 488
  7. 2024From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipelinecited by 484
  8. 2025Video-R1: Reinforcing Video Reasoning in MLLMscited by 464
  9. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Modelcited by 450
  10. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervisioncited by 426
  11. 2019TabFact: A Large-scale Dataset for Table-based Fact Verificationcited by 425
  12. 2025The Lessons of Developing Process Reward Models in Mathematical Reasoningcited by 402
  13. 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRscited by 373
  14. 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approachcited by 372
  15. 2025Accelerating scientific discovery with Co-Scientistcited by 353
  16. 2024ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Searchcited by 338
  17. 2020XCOPA: A Multilingual Dataset for Causal Commonsense Reasoningcited by 330
  18. 2024DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligencecited by 325
  19. 2025Demystifying Long Chain-of-Thought Reasoning in LLMscited by 323
  20. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 323
  21. 2024Graph Retrieval-Augmented Generation: A Surveycited by 320
  22. 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 308
  23. 2024CRUXEval: A Benchmark for Code Reasoning, Understanding and Executioncited by 294
  24. 2025Chain of Draft: Thinking Faster by Writing Lesscited by 290
  25. 2024We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?cited by 281
  26. 2024Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Modelscited by 260
  27. 2024HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMscited by 257
  28. 2025SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolutioncited by 246
  29. 2024Iterative Reasoning Preference Optimizationcited by 244
  30. 2024Token-Budget-Aware LLM Reasoningcited by 242
  31. 2025Training Language Models to Reason Efficientlycited by 242
  32. 2024VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documentscited by 237
  33. 2022DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Modelscited by 230
  34. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 215
  35. 2024Chain-of-Thought Reasoning Without Promptingcited by 208
  36. 2025KernelBench: Can LLMs Write Efficient GPU Kernels?cited by 206
  37. 2024Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judgecited by 202
  38. 2024RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMscited by 200

50 most cited of 226 All 226 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024LLM Reasoners: New Evaluation, Library, and Analysis of Step-by-Step Reasoning with Large Language Models
  2. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systems
  3. 2025Fast, Slow, and Tool-augmented Thinking for LLMs: A Review
  4. 2025Advancing Reasoning in Large Language Models: Promising Methods and Approaches
  5. 2025Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMs
  6. 2026A Comprehensive Evaluation of LLM Reasoning: From Single-Model to Multi-Agent Paradigms
  7. 2025Mapping the Minds of LLMs: A Graph-Based Analysis of Reasoning LLM
  8. 2025Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning