Data to 7 Oct 2026

A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systems

arXiv 2504.09037 ↗First posted 12 Apr 2025cs.AI cs.CLcited by 108

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more

LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Reasoning is a fundamental cognitive process that enables logical inference, problem-solving, and decision-making. With the rapid advancement of large language models (LLMs), reasoning has emerged as a key capability that distinguishes advanced AI systems from conventional models that empower chatbots. In this survey, we categorize existing methods along two orthogonal dimensions: (1) Regimes, which define the stage at which reasoning is achieved (either at inference time or through dedicated training); and (2) Architectures, which determine the components involved in the reasoning process, distinguishing between standalone LLMs and agentic compound systems that incorporate external tools, and multi-agent collaborations. Within each dimension, we analyze two key perspectives: (1) Input level, which focuses on techniques that construct high-quality prompts that the LLM condition on; and (2) Output level, which methods that refine multiple sampled candidates to enhance reasoning quality. This categorization provides a systematic understanding of the evolving landscape of LLM reasoning, highlighting emerging trends such as the shift from inference-scaling to learning-to-reason (e.g., DeepSeek-R1), and the transition to agentic workflows (e.g., OpenAI Deep Research, Manus Agent). Additionally, we cover a broad spectrum of learning algorithms, from supervised fine-tuning to reinforcement learning such as PPO and GRPO, and the training of reasoners and verifiers. We also examine key designs of agentic workflows, from established patterns like generator-evaluator and LLM debate to recent innovations. ...

Key facts. Cited by 108 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 226 of its 466 references that are in the map. First posted on arXiv 12 Apr 2025.

Builds on

226 of 466 references.

Closest ancestors · 24

Same direction, newest first

12 more

Other references · 202

2020Language Models are Few-Shot Learnerscited by 32,199
2022Training language models to follow instructions with human feedbackcited by 15,712
2017Proximal Policy Optimization Algorithmscited by 14,441
2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
2021Training Verifiers to Solve Math Word Problemscited by 9,743
2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
2021Program Synthesis with Large Language Modelscited by 3,509
2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
2022Emergent Abilities of Large Language Modelscited by 2,483

38 more

2019Fine-Tuning Language Models from Human Preferencescited by 2,124
2024Qwen2 Technical Reportcited by 2,076
2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 951
2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 946
2022In-context Learning and Induction Headscited by 930
2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
2022Solving math word problems with process- and outcome-based feedbackcited by 871
2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 763
2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 752
2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 743
2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 724
2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 724
2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
2024Self-Rewarding Language Modelscited by 654
2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 618
2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMscited by 564
2025LIMO: Less is More for Reasoningcited by 488
2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 483
2025Process Reinforcement through Implicit Rewardscited by 478
2024Generative Verifiers: Reward Modeling as Next-Token Predictioncited by 465
2024Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Modelscited by 431
2024Improve Mathematical Reasoning in Language Models by Automated Process Supervisioncited by 426
2023Cognitive Architectures for Language Agentscited by 422
2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Modelscited by 417
2024Training Language Models to Self-Correct via Reinforcement Learningcited by 405
2025The Lessons of Developing Process Reward Models in Mathematical Reasoningcited by 402
2022Self-critiquing models for assisting human evaluatorscited by 372
2019Explain Yourself! Leveraging Language Models for Commonsense Reasoningcited by 367
2024ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Searchcited by 338
2024DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligencecited by 325
2025Demystifying Long Chain-of-Thought Reasoning in LLMscited by 323
2024Skywork-Reward: Bag of Tricks for Reward Modeling in LLMscited by 322
2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 316
2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 308

50 most cited of 202 All 202 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 62; 2026: 46.

108 papers.

Sort:

By area:

2025AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challengescited by 167
2025TRAIL: Trace Reasoning and Agentic Issue Localizationcited by 98
2025Learning What Reinforcement Learning Can't: Interleaved Online Fine-Tuning for Hardest Questionscited by 95
2025VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Usecited by 92
2025Don't Overthink it. Preferring Shorter Thinking Chains for Improved LLM Reasoningcited by 82
2025Safety in Large Reasoning Models: A Surveycited by 58
2025ReAgent-V: A Reward-Driven Multi-Agent Framework for Video Understandingcited by 50
2026Mem-Gallery: Benchmarking Multimodal Long-Term Conversational Memory for MLLM Agentscited by 49
2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 47
2025Evaluating Judges as Evaluators: The JETTS Benchmark of LLM-as-Judges as Test-Time Scaling Evaluatorscited by 42
2025Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AIcited by 39
2025Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecyclecited by 34

38 more

2025Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarkscited by 31
2025FlashThink: An Early Exit Method For Efficient Reasoningcited by 28
2025LLM-based Agentic Reasoning Frameworks: A Survey from Methods to Scenarioscited by 25
2026SkillOrchestra: Learning to Route Agents via Skill Transfercited by 22
2025A Survey of Reasoning and Agentic Systems in Time Series with Large Language Modelscited by 21
2025A Survey on Agentic Multimodal Large Language Modelscited by 19
2025Hard2Verify: A Step-Level Verification Benchmark for Open-Ended Frontier Mathcited by 19
2025Variation in Verification: Understanding Verification Dynamics in Large Language Modelscited by 18
2025Plan Then Action:High-Level Planning Guidance Reinforcement Learning for LLM Reasoningcited by 17
2025Joint Evaluation of Answer and Reasoning Consistency for Hallucination Detection in Large Reasoning Modelscited by 13
2025Traffic-R1: Reinforced LLMs Bring Human-Like Reasoning to Traffic Signal Control Systemscited by 12
2025From LLMs to MLLMs to Agents: A Survey of Emerging Paradigms in Jailbreak Attacks and Defenses within LLM Ecosystemcited by 11
2025Not All Errors Are Created Equal: ASCoT Addresses Late-Stage Fragility in Efficient LLM Reasoningcited by 11
2025Sycophancy Mitigation Through Reinforcement Learning with Uncertainty-Aware Adaptive Reasoning Trajectoriescited by 10
2025J4R: Learning to Judge with Equivalent Initial State Group Relative Policy Optimizationcited by 9
2025HiMA-Ecom: Enabling Joint Training of Hierarchical Multi-Agent E-commerce Assistantscited by 9
2025Reasoning-Enhanced Large Language Models for Molecular Property Predictioncited by 9
2025FunReason-MT Technical Report: Advanced Data Synthesis Solution for Real-world Multi-Turn Tool-usecited by 9
2025Don't Take the Premise for Granted: Evaluating the Premise Critique Ability of Large Language Modelscited by 8
2025Lexical Hints of Accuracy in LLM Reasoning Chainscited by 8
2025S-DAG: A Subject-Based Directed Acyclic Graph for Multi-Agent Heterogeneous Reasoningcited by 8
2025Longer Context, Deeper Thinking: Uncovering the Role of Long-Context Ability in Reasoningcited by 7
2025Beyond Accuracy: Dissecting Mathematical Reasoning for LLMs Under Reinforcement Learningcited by 7
2025Thinking Longer, Not Always Smarter: Evaluating LLM Capabilities in Hierarchical Legal Reasoningcited by 7
2025LocalSearchBench: Benchmarking Agentic Search in Real-World Local Life Servicescited by 7
2025GLARE: Agentic Reasoning for Legal Judgment Predictioncited by 6
2025SSR: Socratic Self-Refine for Large Language Model Reasoningcited by 6
2026Generalization in Online Reinforcement Learning for Mobile Agentscited by 6
2025Foundational Automatic Evaluators: Scaling Multi-Task Generative Evaluator Training for Reasoning-Centric Domainscited by 5
2025AgentSense: LLMs Empower Generalizable and Explainable Web-Based Participatory Urban Sensingcited by 5
2026FROST: Filtering Reasoning Outliers with Attention for Efficient Reasoningcited by 5
2026LegalGraphRAG: Multi-Agent Graph Retrieval-Augmented Generation for Reliable Legal Reasoningcited by 5
2026ReNIO: Reweighting Negative Trajectory Importance for LLM On-Policy Distillationcited by 5
2026AI4AI at Test-Time: Strong-to-Weak Capability Transfer via Harnessescited by 5
2025Far from the Shallow: Brain-Predictive Reasoning Embedding through Residual Disentanglementcited by 4
2026On the Power of (Approximate) Reward Models for Inference-Time Scalingcited by 4
2026Pencil Puzzle Bench: A Benchmark for Multi-Step Verifiable Reasoningcited by 4
2026Zipage: Maintain High Request Concurrency for LLM Reasoning through Compressed PagedAttentioncited by 4

50 most cited of 108 All 108 (JSON)

Similar, not cited

Close in wording, no citation either way