The Periodic Table of LLM Reasoning: A Structured Survey of Reasoning Paradigms, Methods, and Failure Modes
On the map: LLM reasoning → Reasoning benchmarks and evaluation
also in 1 more
Abstract (arXiv)
Reasoning has become central to how Large Language Models (LLMs) are evaluated and interpreted, spanning Chain-of-Thought (CoT), mathematical problem-solving, multi-hop question answering, code generation, retrieval-augmented reasoning, tool use, and multimodal decision-making. In this survey, we introduce the Periodic Table of LLM Reasoning, a framework organizing 300+ recent papers by reasoning paradigm, methodological mechanism, evaluation setting, and failure mode. We classify LLM reasoning into nine paradigms: Chain-of-Thought, Multi-Hop, Mathematical, Commonsense, Visual and Temporal, Code and Algorithmic, Retrieval-Augmented, Tool-Augmented or Agentic, and Reinforcement Learning-based reasoning. For each, we review approaches, including prompting, architectural interventions, supervised fine-tuning, verifier-guided inference, reward modeling, retrieval, tool interfaces, agentic workflows, and benchmark design. We argue that LLM reasoning is not a single emergent capability but a family of scaffolded behaviors shaped by model scale, task structure, external memory, supervision, and evaluation protocols. We synthesize recurring failure modes, including hallucinated reasoning, brittle multi-step inference, spurious rationales, weak causal grounding, poor out-of-distribution generalization, benchmark contamination, and unreliable self-verification. Progress is difficult to compare across paradigms because gains may arise from prompting, retrieval, verifier design, or benchmark structure rather than general reasoning ability. The survey connects methods to their assumptions, strengths, and failure modes, providing a reference map of the field and a diagnostic framework for future work. We conclude that robust LLM reasoning will require meta-reasoning, multimodal and temporal grounding, adaptive tool use, and principled evaluation under distribution shift.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 262 of its 342 references that are in the map. First posted on arXiv 9 Jun 2026.
Builds on
262 of 342 references.
Closest ancestors · 36
Same direction, newest first
- 2025mSCoRe: a Multilingual and Scalable Benchmark for Skill-based Commonsense Reasoning13 Aug
- 2025GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models8 Aug
- 2025MultiNRC: A Challenging and Native Multilingual Reasoning Evaluation Benchmark for LLMs23 Jul
- 2025Who Benchmarks the Benchmarks? Towards Comprehensive Evaluation of Commonsense Reasoning Benchmarks10 Apr
- 2025DeepSeek-R1 Thoughtology: Let's think about LLM Reasoning2 Apr
- 2025JudgeLRM: Large Reasoning Models as a Judge31 Mar
- 2025Landscape of Thoughts: Visualizing the Reasoning Process of Large Language Models28 Mar
- 2025I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoders24 Mar
- 2025Towards Better Understanding of Program-of-Thought Reasoning in Cross-Lingual and Multilingual Environments25 Feb
- 2025Does Reasoning Introduce Bias? A Study of Social Bias Evaluation and Mitigation in LLM Reasoning21 Feb
- 2025A Survey of Multilingual Reasoning in Language Models13 Feb
- 2025Adapting Language-Specific LLMs to a Reasoning Model in One Day via Model Merging -- An Open Recipe13 Feb
24 more
- 2025Great Models Think Alike and this Undermines AI Oversight6 Feb
- 2025ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoning3 Feb
- 2025Large Language Models Think Too Fast To Explore Effectively29 Jan
- 2025Reasoning Language Models: A Blueprint20 Jan
- 2025Multiple Choice Questions: Reasoning Makes Large Language Models (LLMs) More Self-Confident, Especially When They are Wrong16 Jan
- 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Models16 Jan
- 2025SLAM: Towards Efficient Multilingual Reasoning via Selective Language Alignment7 Jan
- 2024LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks19 Dec
- 2024Are Your LLMs Capable of Stable Reasoning?17 Dec
- 2024SocialGPT: Prompting LLMs for Social Relation Reasoning via Greedy Segment Optimization28 Oct
- 2024Iteration of Thought: Leveraging Inner Dialogue for Autonomous Large Language Model Reasoning19 Sep
- 2024Large Language Models Are Cross-Lingual Knowledge-Free Reasoners24 Jun
- 2024BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack14 Jun
- 2024MindMerger: Efficient Boosting LLM Reasoning in non-English Languages27 May
- 2024Advancing LLM Reasoning Generalists with Preference Trees2 Apr
- 2024Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Models19 Feb
- 2024Understanding Reasoning Ability of Language Models From the Perspective of Reasoning Paths Aggregation5 Feb
- 2024Efficient Tool Use with Chain-of-Abstraction Reasoning30 Jan
- 2024LangBridge: Multilingual Reasoning Without Multilingual Supervision19 Jan
- 2024MAPO: Advancing Multilingual Reasoning through Multilingual Alignment-as-Preference Optimization12 Jan
- 2023Understanding Social Reasoning in Language Models with Language Models21 Jun
- 2023Large Language Models Are Not Strong Abstract Reasoners31 May
- 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity8 Feb
- 2022ReAct: Synergizing Reasoning and Acting in Language Models6 Oct
Other references · 226
- 2023GPT-4 Technical Reportcited by 18,132
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
- 2022Emergent Abilities of Large Language Modelscited by 2,483
- 2022Language Models (Mostly) Know What They Knowcited by 1,783
- 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
- 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 740
- 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 573
38 more
- 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMscited by 564
- 2024GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Modelscited by 533
- 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 527
- 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 518
- 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 501
- 2025LIMO: Less is More for Reasoningcited by 488
- 2024From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipelinecited by 484
- 2025Video-R1: Reinforcing Video Reasoning in MLLMscited by 464
- 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Modelcited by 450
- 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervisioncited by 426
- 2019TabFact: A Large-scale Dataset for Table-based Fact Verificationcited by 425
- 2025The Lessons of Developing Process Reward Models in Mathematical Reasoningcited by 402
- 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRscited by 373
- 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approachcited by 372
- 2025Accelerating scientific discovery with Co-Scientistcited by 353
- 2024ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Searchcited by 338
- 2020XCOPA: A Multilingual Dataset for Causal Commonsense Reasoningcited by 330
- 2024DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligencecited by 325
- 2025Demystifying Long Chain-of-Thought Reasoning in LLMscited by 323
- 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 323
- 2024Graph Retrieval-Augmented Generation: A Surveycited by 320
- 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 308
- 2024CRUXEval: A Benchmark for Code Reasoning, Understanding and Executioncited by 294
- 2025Chain of Draft: Thinking Faster by Writing Lesscited by 290
- 2024We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?cited by 281
- 2024Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Modelscited by 260
- 2024HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMscited by 257
- 2025SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolutioncited by 246
- 2024Iterative Reasoning Preference Optimizationcited by 244
- 2024Token-Budget-Aware LLM Reasoningcited by 242
- 2025Training Language Models to Reason Efficientlycited by 242
- 2024VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documentscited by 237
- 2022DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Modelscited by 230
- 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 215
- 2024Chain-of-Thought Reasoning Without Promptingcited by 208
- 2025KernelBench: Can LLMs Write Efficient GPU Kernels?cited by 206
- 2024Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judgecited by 202
- 2024RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMscited by 200
50 most cited of 226 All 226 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024LLM Reasoners: New Evaluation, Library, and Analysis of Step-by-Step Reasoning with Large Language Models
- 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systems
- 2025Fast, Slow, and Tool-augmented Thinking for LLMs: A Review
- 2025Advancing Reasoning in Large Language Models: Promising Methods and Approaches
- 2025Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMs
- 2026A Comprehensive Evaluation of LLM Reasoning: From Single-Model to Multi-Agent Paradigms
- 2025Mapping the Minds of LLMs: A Graph-Based Analysis of Reasoning LLM
- 2025Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning