mlsanityData to 7 Oct 2026

Multi-Step Reasoning with Large Language Models, a Survey

arXiv 2407.11511 ↗First posted 16 Jul 2024cs.AI cs.CL cs.LGcited by 140

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Large language models (LLMs) with billions of parameters exhibit in-context learning abilities, enabling few-shot learning on tasks that the model was not specifically trained for. Traditional models achieve breakthrough performance on language tasks, but do not perform well on basic reasoning benchmarks. However, a new in-context learning approach, Chain-of-thought, has demonstrated strong multi-step reasoning abilities on these benchmarks. The research on LLM reasoning abilities started with the question whether LLMs can solve grade school math word problems, and has expanded to other tasks in the past few years. This article reviews the field of multi-step reasoning with LLMs. We propose a taxonomy that identifies different ways to generate, evaluate, and control multi-step reasoning. We provide an in-depth coverage of core approaches and open problems, and we propose a research agenda for the near future. We find that multi-step reasoning approaches have progressed beyond math word problems, and can now successfully solve challenges in logic, combinatorial games, and robotics, sometimes by first generating code that is then executed by external tools. Many studies in multi-step methods use reinforcement learning for finetuning, external optimization loops, in-context reinforcement learning, and self-reflection.

Key facts. Cited by 140 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 129 of its 199 references that are in the map. First posted on arXiv 16 Jul 2024.

Builds on

129 of 199 references.

Closest ancestors · 14

Same direction, newest first

  1. 2024Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey2 Apr
  2. 2023Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks5 Jul
  3. 2023Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning20 May
  4. 2023Evaluating the Logical Reasoning Ability of ChatGPT and GPT-47 Apr
  5. 2023Natural Language Reasoning, A Survey26 Mar
  6. 2022Towards Reasoning in Large Language Models: A Survey20 Dec
  7. 2022PAL: Program-aided Language Models18 Nov
  8. 2022Explanations from Large Language Models Make Small Reasoners Better13 Oct
  9. 2022Measuring and Narrowing the Compositionality Gap in Language Models7 Oct
  10. 2022ReAct: Synergizing Reasoning and Acting in Language Models6 Oct
  11. 2022Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought3 Oct
  12. 2022FOLIO: Natural Language Reasoning with First-Order Logic2 Sep
2 more
  1. 2022Making Large Language Models Better Reasoners with Step-Aware Verifier6 Jun
  2. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Models30 Nov

Other references · 115

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  7. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  8. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  9. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  10. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  11. 2020Scaling Laws for Neural Language Modelscited by 6,411
  12. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
38 more
  1. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  2. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  4. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  5. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  7. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  8. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  9. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  10. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  11. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  12. 2023A Survey of Large Language Modelscited by 2,537
  13. 2022Emergent Abilities of Large Language Modelscited by 2,483
  14. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  15. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  16. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  17. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  18. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  19. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  20. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  21. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  22. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  23. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  24. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  25. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  26. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  27. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  28. 2017Mixed Precision Trainingcited by 1,135
  29. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  30. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  31. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  32. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  33. 2023Teaching Large Language Models to Self-Debugcited by 963
  34. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 960
  35. 2020Meta-Learning in Neural Networks: A Surveycited by 944
  36. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  37. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  38. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784

50 most cited of 115 All 115 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 95; 2026: 38.

140 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  2. 2025Agentic Large Language Models, a surveycited by 105
  3. 2025STP: Self-play LLM Theorem Provers with Iterative Conjecturing and Provingcited by 77
  4. 2024GEM: A Generalizable Ego-Vision Multimodal World Model for Fine-Grained Ego-Motion, Object Dynamics, and Scene Composition Controlcited by 76
  5. 2024Beyond Examples: High-level Automated Reasoning Paradigm in In-Context Learning via MCTScited by 52
  6. 2025Logical Reasoning in Large Language Models: A Surveycited by 52
  7. 2024Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspectivecited by 49
  8. 2025Dynamic Parallel Tree Search for Efficient LLM Reasoningcited by 42
  9. 2025DeepRAG: Thinking to Retrieve Step by Step for Large Language Modelscited by 37
  10. 2025Navigate the Unknown: Enhancing LLM Reasoning with Intrinsic Motivation Guided Explorationcited by 33
  11. 2025Table-Critic: A Multi-Agent Framework for Collaborative Criticism and Refinement in Table Reasoningcited by 32
  12. 2025LLM-Powered GUI Agents in Phone Automation: Surveying Progress and Prospectscited by 31
38 more
  1. 2025Assessing Judging Bias in Large Reasoning Models: An Empirical Studycited by 30
  2. 2025Towards Reasoning Ability of Small Language Modelscited by 29
  3. 2025Entropy-based Exploration Conduction for Multi-step Reasoningcited by 28
  4. 2025FinanceReasoning: Benchmarking Financial Numerical Reasoning More Credible, Comprehensive and Challengingcited by 27
  5. 2025Understanding the Uncertainty of LLM Explanations: A Perspective Based on Reasoning Topologycited by 24
  6. 2025Nuclear Deployed: Analyzing Catastrophic Risks in Decision-making of Autonomous LLM Agentscited by 23
  7. 2025ShadowCoT: Cognitive Hijacking for Stealthy Reasoning Backdoors in LLMscited by 23
  8. 2025Thought Manipulation: External Thought Can Be Efficient for Large Reasoning Modelscited by 23
  9. 2025To Think or Not to Think: Exploring the Unthinking Vulnerability in Large Reasoning Modelscited by 22
  10. 2025Towards a Design Guideline for RPA Evaluation: A Survey of Large Language Model-Based Role-Playing Agentscited by 22
  11. 2025Mind with Eyes: from Language Reasoning to Multimodal Reasoningcited by 22
  12. 2025Advancing Tool-Augmented Large Language Models via Meta-Verification and Reflection Learningcited by 22
  13. 2026Large Language Model Reasoning Failurescited by 20
  14. 2025Improving RL Exploration for LLM Reasoning through Retrospective Replaycited by 19
  15. 2025How does Chain of Thought Think? Mechanistic Interpretability of Chain-of-Thought Reasoning with Sparse Autoencodingcited by 19
  16. 2025Commonsense Reasoning in Arab Culturecited by 16
  17. 2025Reasoning Beyond Limits: Advances and Open Problems for LLMscited by 16
  18. 2025Spatial Reasoning with Denoising Modelscited by 15
  19. 2025CoRT: Code-integrated Reasoning within Thinkingcited by 15
  20. 2025A Survey of Attacks on Large Language Modelscited by 13
  21. 2025Reinforcement Fine-Tuning Powers Reasoning Capability of Multimodal Large Language Modelscited by 13
  22. 2025VERUS-LM: a Versatile Framework for Combining LLMs with Symbolic Reasoningcited by 12
  23. 2025Grokking in the Wild: Data Augmentation for Real-World Multi-Hop Reasoning with Transformerscited by 12
  24. 2025Reinforced Reasoning for Embodied Planningcited by 12
  25. 2025Learning from Diverse Reasoning Paths with Routing and Collaborationcited by 12
  26. 2025A Survey on Feedback-based Multi-step Reasoning for Large Language Models on Mathematicscited by 11
  27. 2025Evaluating System 1 vs. 2 Reasoning Approaches for Zero-Shot Time Series Forecasting: A Benchmark and Insightscited by 11
  28. 2024Reversal of Thought: Enhancing Large Language Models with Preference-Guided Reverse Reasoning Warm-upcited by 10
  29. 2025Meta-Thinking in LLMs via Multi-Agent Reinforcement Learning: A Surveycited by 10
  30. 2025BioHopR: A Benchmark for Multi-Hop, Multi-Answer Reasoning in Biomedical Domaincited by 10
  31. 2026Dynamic Skill Lifecycle Management for Agentic Reinforcement Learningcited by 10
  32. 2025CrossWordBench: Evaluating the Reasoning Capabilities of LLMs and LVLMs with Controllable Puzzle Generationcited by 9
  33. 2025SEM: Reinforcement Learning for Search-Efficient Large Language Modelscited by 9
  34. 2025Failure Modes in LLM Systems: A System-Level Taxonomy for Reliable AI Applicationscited by 9
  35. 2026SELAUR: Self Evolving LLM Agent via Uncertainty-aware Rewardscited by 9
  36. 2024Length Controlled Generation for Black-box LLMscited by 8
  37. 2025Thinking Preference Optimizationcited by 8
  38. 2025MemeIntel: Explainable Detection of Propagandistic and Hateful Memescited by 8

50 most cited of 140 All 140 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025A Survey on Large Language Models for Mathematical Reasoning
  2. 2023Learning Multi-Step Reasoning by Solving Arithmetic Tasks
  3. 2024Step Guided Reasoning: Improving Mathematical Reasoning using Guidance Generation and Step Reasoning
  4. 2025LogicPuzzleRL: Cultivating Robust Mathematical Reasoning in LLMs via Reinforcement Learning
  5. 2025Advancing Reasoning in Large Language Models: Promising Methods and Approaches
  6. 2026Opening the Black Box: A Survey on the Mechanisms of Multi-Step Reasoning in Large Language Models
  7. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Models
  8. 2024Benchmarking Large Language Models for Math Reasoning Tasks