mlsanityData to 14 Oct 2026

A Tutorial on LLM Reasoning: Relevant Methods behind ChatGPT o1

arXiv 2502.10867 ↗First posted 15 Feb 2025cs.AI cs.CLcited by 25

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

OpenAI o1 has shown that applying reinforcement learning to integrate reasoning steps directly during inference can significantly improve a model's reasoning capabilities. This result is exciting as the field transitions from the conventional autoregressive method of generating answers to a more deliberate approach that models the slow-thinking process through step-by-step reasoning training. Reinforcement learning plays a key role in both the model's training and decoding processes. In this article, we present a comprehensive formulation of reasoning problems and investigate the use of both model-based and model-free approaches to better support this slow-thinking framework.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 21 of its 35 references that are in the map. First posted on arXiv 15 Feb 2025.

Builds on

21 of 35 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug
  2. 2024Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models1 Aug
  3. 2023Think before you speak: Training Language Models With Pause Tokens3 Oct

Other references · 18

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  6. 2020Scaling Laws for Neural Language Modelscited by 6,586
  7. 2023Let's Verify Step by Stepcited by 3,789
  8. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  9. 2022Solving math word problems with process- and outcome-based feedbackcited by 901
  10. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 886
  11. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 734
  12. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervisioncited by 441
6 more
  1. 2022Making Large Language Models Better Reasoners with Step-Aware Verifiercited by 233
  2. 2024Token-level Direct Preference Optimizationcited by 135
  3. 2020Limitations of Autoregressive Models and Their Alternativescited by 55
  4. 2024Multi-step Problem Solving Through a Verifier: An Empirical Analysis on Model-induced Process Supervisioncited by 52
  5. 2023Critique Ability of Large Language Modelscited by 20
  6. 2023Why Can Large Language Models Generate Correct Chain-of-Thoughts?cited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 15; 2026: 10.

25 papers.

Sort:

By area:

  1. 2025Mind with Eyes: from Language Reasoning to Multimodal Reasoningcited by 22
  2. 2025AdaReasoner: Adaptive Reasoning Enables More Flexible Thinking in Large Language Modelscited by 17
  3. 2025A Practical Guide for Designing, Developing, and Deploying Production-Grade Agentic AI Workflowscited by 15
  4. 2025Reinforcement Fine-Tuning Powers Reasoning Capability of Multimodal Large Language Modelscited by 14
  5. 2025Towards Responsible and Explainable AI Agents with Consensus-Driven Reasoningcited by 14
  6. 2025Standardization of Psychiatric Diagnoses -- Role of Fine-tuned LLM Consortium and OpenAI-gpt-oss Reasoning LLM Enabled Decision Support Systemcited by 12
  7. 2025QUITE: A Query Rewrite System Beyond Rules with LLM Agentscited by 8
  8. 2025Memento 2: Learning by Stateful Reflective Memorycited by 6
  9. 2026A Practical Guide to Agentic AI Transition in Organizationscited by 5
  10. 2026MemSearch-o1: Empowering Large Language Models with Reasoning-Aligned Memory Growth in Agentic Searchcited by 5
  11. 2025Self-Verifying Reflection Helps Transformers with CoT Reasoningcited by 4
  12. 2025Evaluating the Efficacy of LLM-Based Reasoning for Multiobjective HPC Job Schedulingcited by 3
13 more
  1. 2025DeepSeek: Paradigm Shifts and Technical Evolution in Large AI Modelscited by 3
  2. 2025Standardization of Neuromuscular Reflex Analysis -- Role of Fine-Tuned Vision-Language Model Consortium and OpenAI gpt-oss Reasoning LLM Enabled Decision Support Systemcited by 3
  3. 2025Semi-structured LLM Reasoners Can Be Rigorously Auditedcited by 2
  4. 2026Flowr -- Scaling Up Retail Supply Chain Operations Through Agentic AI in Large Scale Supermarket Chainscited by 2
  5. 2025Probing the "Psyche'' of Large Reasoning Models: Understanding Through a Human Lenscited by 1
  6. 2026UniScale: Adaptive Unified Inference Scaling via Online Joint Optimization of Model Routing and Test-Time Scalingcited by 1
  7. 2025Semantic Distance Measurement based on Multi-Kernel Gaussian Processescited by 0
  8. 2026Phylogenetic Tree Inference with Tropical Axial Attentioncited by 0
  9. 2026Train the Trainers -- An Agentic AI Framework for Peer-Based Mental Health Support in Battlefield Environmentscited by 0
  10. 2026IsoSci: A Benchmark of Isomorphic Cross-Domain Science Problems for Evaluating Reasoning versus Knowledge Retrieval in LLMscited by 0
  11. 2026BiasTrace: Linking Reasoning Behaviours to Biased Outputs in LLMscited by 0
  12. 2026Large Discovery Models: Empirically-grounded Model-Based Open-Ended Searchcited by 0
  13. 2026BaseCamp --- An Agentic AI Framework for Automating DNA Sequencing Data Pipelinescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Survey of Slow Thinking-based Reasoning LLMs using Reinforced Learning and Inference-time Scaling Law
  2. 2024OpenR: An Open Source Framework for Advanced Reasoning with Large Language Models
  3. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systems
  4. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models
  5. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Models
  6. 2024A Comparative Study on Reasoning Patterns of OpenAI's o1 Model
  7. 2025Thinkless: LLM Learns When to Think
  8. 2025Logical Reasoning in Large Language Models: A Survey