mlsanityData to 7 Oct 2026

LLM Post-Training: A Deep Dive into Reasoning Large Language Models

arXiv 2502.21321 ↗First posted 28 Feb 2025cs.CL cs.CVcited by 129

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Large Language Models (LLMs) have transformed the natural language processing landscape and brought to life diverse applications. Pretraining on vast web-scale data has laid the foundation for these models, yet the research community is now increasingly shifting focus toward post-training techniques to achieve further breakthroughs. While pretraining provides a broad linguistic foundation, post-training methods enable LLMs to refine their knowledge, improve reasoning, enhance factual accuracy, and align more effectively with user intents and ethical considerations. Fine-tuning, reinforcement learning, and test-time scaling have emerged as critical strategies for optimizing LLMs performance, ensuring robustness, and improving adaptability across various real-world tasks. This survey provides a systematic exploration of post-training methodologies, analyzing their role in refining LLMs beyond pretraining, addressing key challenges such as catastrophic forgetting, reward hacking, and inference-time trade-offs. We highlight emerging directions in model alignment, scalable adaptation, and inference-time reasoning, and outline future research directions. We also provide a public repository to continually track developments in this fast-evolving field: https://github.com/mbzuai-oryx/Awesome-LLM-Post-training.

Key facts. Cited by 129 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 287 of its 385 references that are in the map. First posted on arXiv 28 Feb 2025.

Builds on

287 of 385 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024Q*: Improving Multi-step Reasoning for LLMs with Deliberative Planning20 Jun
  2. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools18 Jun
  3. 2024LLM Reasoners: New Evaluation, Library, and Analysis of Step-by-Step Reasoning with Large Language Models8 Apr
  4. 2023PaLM 2 Technical Report17 May
  5. 2022Thinking Fast and Slow in Large Language Models10 Dec
  6. 2022Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought3 Oct
  7. 2022FOLIO: Natural Language Reasoning with First-Order Logic2 Sep

Other references · 280

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2022Training language models to follow instructions with human feedbackcited by 15,712
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  7. 2017Proximal Policy Optimization Algorithmscited by 14,441
  8. 2024The Llama 3 Herd of Modelscited by 14,192
  9. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  10. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  11. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  12. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
38 more
  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2020Scaling Laws for Neural Language Modelscited by 6,411
  6. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  7. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  8. 2024Qwen2.5 Technical Reportcited by 4,713
  9. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  10. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  11. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  12. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  13. 2023Let's Verify Step by Stepcited by 3,598
  14. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  15. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  16. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  17. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  18. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  19. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  20. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  21. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  22. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  23. 2020Learning to summarize from human feedbackcited by 2,562
  24. 2023A Survey of Large Language Modelscited by 2,537
  25. 2022Emergent Abilities of Large Language Modelscited by 2,483
  26. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  27. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  28. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  29. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  30. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  31. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  32. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  33. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  34. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  35. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  36. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  37. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  38. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800

50 most cited of 280 All 280 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 80; 2026: 49.

129 papers.

Sort:

By area:

  1. 2025ToolRL: Reward is All Tool Learning Needscited by 342
  2. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 195
  3. 2025WebDancer: Towards Autonomous Information Seeking Agencycited by 180
  4. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  5. 2025Reason-RFT: Reinforcement Fine-Tuning for Visual Reasoning of Vision Language Modelscited by 92
  6. 2025Fin-R1: A Large Language Model for Financial Reasoning through Reinforcement Learningcited by 82
  7. 2025SpatialReasoner: Towards Explicit and Generalizable 3D Spatial Reasoningcited by 65
  8. 2025Look-Back: Implicit Visual Re-focusing in MLLM Reasoningcited by 54
  9. 2025Think Before Recommend: Unleashing the Latent Reasoning Power for Sequential Recommendationcited by 50
  10. 2025Reinforcement Fine-Tuning Naturally Mitigates Forgetting in Continual Post-Trainingcited by 42
  11. 2025Self-Evolving Multi-Agent Simulations for Realistic Clinical Interactionscited by 40
  12. 2025Arctic-Text2SQL-R1: Simple Rewards, Strong Reasoning in Text-to-SQLcited by 38
38 more
  1. 2025Enhancing LLM Reasoning with Iterative DPO: A Comprehensive Empirical Investigationcited by 36
  2. 2025Learning to Reason for Long-Form Story Generationcited by 34
  3. 2025Contextual Integrity in LLMs via Reasoning and Reinforcement Learningcited by 30
  4. 2025Self-Improving LLM Agents at Test-Timecited by 30
  5. 2025Quagmires in SFT-RL Post-Training: When High SFT Scores Mislead and What to Use Insteadcited by 24
  6. 2025Time-R1: Towards Comprehensive Temporal Reasoning in LLMscited by 23
  7. 2025Graphs Meet AI Agents: Taxonomy, Progress, and Future Opportunitiescited by 23
  8. 2025Trade-offs in Large Reasoning Models: An Empirical Analysis of Deliberative and Adaptive Reasoning over Foundational Capabilitiescited by 20
  9. 2025Improving RL Exploration for LLM Reasoning through Retrospective Replaycited by 19
  10. 2025Your Pre-trained LLM is Secretly an Unsupervised Confidence Calibratorcited by 19
  11. 2025A Survey of Slow Thinking-based Reasoning LLMs using Reinforced Learning and Inference-time Scaling Lawcited by 18
  12. 2025Agent-X: Evaluating Deep Multimodal Reasoning in Vision-Centric Agentic Taskscited by 17
  13. 2026ROSD: Reflective On-Policy Self-Distillation for Language Model Reasoning across Domainscited by 17
  14. 2025Reasoning Beyond Limits: Advances and Open Problems for LLMscited by 16
  15. 2025Co-Reinforcement Learning for Unified Multimodal Understanding and Generationcited by 15
  16. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 14
  17. 2025CoIn: Counting the Invisible Reasoning Tokens in Commercial Opaque LLM APIscited by 14
  18. 2025Thinking in Character: Advancing Role-Playing Agents with Role-Aware Reasoningcited by 14
  19. 2025Pre-Trained Policy Discriminators are General Reward Modelscited by 14
  20. 2025BigCharts-R1: Enhanced Chart Reasoning with Visual Reinforcement Finetuningcited by 13
  21. 2025GeoZero: Incentivizing Reasoning from Scratch on Geospatial Scenescited by 13
  22. 2025Aligning Multimodal LLM with Human Preference: A Surveycited by 12
  23. 2025Robustness in Large Language Models: A Survey of Mitigation Strategies and Evaluation Metricscited by 12
  24. 2025Reasoning Model Unlearning: Forgetting Traces, Not Just Answers, While Preserving Reasoning Skillscited by 12
  25. 2025FakeScope: Large Multimodal Expert Model for Transparent AI-Generated Image Forensicscited by 11
  26. 2025AdapThink: Adaptive Thinking Preferences for Reasoning Language Modelcited by 11
  27. 2025From Seeing to Experiencing: Scaling Navigation Foundation Models with Reinforcement Learningcited by 11
  28. 2025InSPO: Unlocking Intrinsic Self-Reflection for LLM Preference Optimizationcited by 11
  29. 2025A Survey of Scaling in Large Language Model Reasoningcited by 10
  30. 2025Exploring the System 1 Thinking Capability of Large Reasoning Modelscited by 10
  31. 2025A Desideratum for Conversational Agents: Capabilities, Challenges, and Future Directionscited by 9
  32. 2025ProtoReasoning: Prototypes as the Foundation for Generalizable Reasoning in LLMscited by 9
  33. 2026Probability-Entropy Calibration: An Elastic Indicator for Adaptive Fine-tuningcited by 9
  34. 2025Steerable Scene Generation with Post Training and Inference-Time Searchcited by 8
  35. 2025Toward Effective Reinforcement Learning Fine-Tuning for Medical VQA in Vision-Language Modelscited by 8
  36. 2025Revisiting Test-Time Scaling: A Survey and a Diversity-Aware Method for Efficient Reasoningcited by 8
  37. 2025Reinforcement Learning Fine-Tuning Enhances Activation Intensity and Diversity in the Internal Circuitry of LLMscited by 8
  38. 2025Video-LMM Post-Training: A Deep Dive into Video Reasoning with Large Multimodal Modelscited by 8

50 most cited of 129 All 129 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Advancing Reasoning in Large Language Models: Promising Methods and Approaches
  2. 2025Temporal Sampling for Forgotten Reasoning in LLMs
  3. 2025DeepDistill: Enhancing LLM Reasoning Capabilities via Large-Scale Difficulty-Graded Data Training
  4. 2024Distilling LLMs' Decomposition Abilities into Compact Language Models
  5. 2025A Survey on Post-training of Large Language Models
  6. 2026Learning, Fast and Slow: Towards LLMs That Adapt Continually
  7. 2024A critical review of methods and challenges in large language models
  8. 2026PRISM: A Unified Framework for Post-Training LLMs Without Verifiable Rewards