# LLM Post-Training: A Deep Dive into Reasoning Large Language Models

arXiv: https://arxiv.org/abs/2502.21321 · HTML: https://mlsanity.com/paper/2502.21321/

## Key facts

- Cited by 135 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 287 of its 385 references that are in the map.
- First posted on arXiv 28 Feb 2025.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 30 of 287)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)

## Built on by (most cited first, 30 of 135)

- [ToolRL: Reward is All Tool Learning Needs](https://mlsanity.com/paper/2504.13958/) (2025)
- [The Landscape of Agentic Reinforcement Learning for LLMs: A Survey](https://mlsanity.com/paper/2509.02547/) (2025)
- [WebDancer: Towards Autonomous Information Seeking Agency](https://mlsanity.com/paper/2505.22648/) (2025)
- [A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment](https://mlsanity.com/paper/2504.15585/) (2025)
- [Reason-RFT: Reinforcement Fine-Tuning for Visual Reasoning of Vision Language Models](https://mlsanity.com/paper/2503.20752/) (2025)
- [Fin-R1: A Large Language Model for Financial Reasoning through Reinforcement Learning](https://mlsanity.com/paper/2503.16252/) (2025)
- [SpatialReasoner: Towards Explicit and Generalizable 3D Spatial Reasoning](https://mlsanity.com/paper/2504.20024/) (2025)
- [Look-Back: Implicit Visual Re-focusing in MLLM Reasoning](https://mlsanity.com/paper/2507.03019/) (2025)
- [Think Before Recommend: Unleashing the Latent Reasoning Power for Sequential Recommendation](https://mlsanity.com/paper/2503.22675/) (2025)
- [Reinforcement Fine-Tuning Naturally Mitigates Forgetting in Continual Post-Training](https://mlsanity.com/paper/2507.05386/) (2025)
- [Self-Evolving Multi-Agent Simulations for Realistic Clinical Interactions](https://mlsanity.com/paper/2503.22678/) (2025)
- [Arctic-Text2SQL-R1: Simple Rewards, Strong Reasoning in Text-to-SQL](https://mlsanity.com/paper/2505.20315/) (2025)
- [Enhancing LLM Reasoning with Iterative DPO: A Comprehensive Empirical Investigation](https://mlsanity.com/paper/2503.12854/) (2025)
- [Learning to Reason for Long-Form Story Generation](https://mlsanity.com/paper/2503.22828/) (2025)
- [Contextual Integrity in LLMs via Reasoning and Reinforcement Learning](https://mlsanity.com/paper/2506.04245/) (2025)
- [Self-Improving LLM Agents at Test-Time](https://mlsanity.com/paper/2510.07841/) (2025)
- [Quagmires in SFT-RL Post-Training: When High SFT Scores Mislead and What to Use Instead](https://mlsanity.com/paper/2510.01624/) (2025)
- [Time-R1: Towards Comprehensive Temporal Reasoning in LLMs](https://mlsanity.com/paper/2505.13508/) (2025)
- [Graphs Meet AI Agents: Taxonomy, Progress, and Future Opportunities](https://mlsanity.com/paper/2506.18019/) (2025)
- [Trade-offs in Large Reasoning Models: An Empirical Analysis of Deliberative and Adaptive Reasoning over Foundational Capabilities](https://mlsanity.com/paper/2503.17979/) (2025)
- [Your Pre-trained LLM is Secretly an Unsupervised Confidence Calibrator](https://mlsanity.com/paper/2505.16690/) (2025)
- [Improving RL Exploration for LLM Reasoning through Retrospective Replay](https://mlsanity.com/paper/2504.14363/) (2025)
- [A Survey of Slow Thinking-based Reasoning LLMs using Reinforced Learning and Inference-time Scaling Law](https://mlsanity.com/paper/2505.02665/) (2025)
- [Agent-X: Evaluating Deep Multimodal Reasoning in Vision-Centric Agentic Tasks](https://mlsanity.com/paper/2505.24876/) (2025)
- [ROSD: Reflective On-Policy Self-Distillation for Language Model Reasoning across Domains](https://mlsanity.com/paper/2605.28014/) (2026)
- [Reasoning Beyond Limits: Advances and Open Problems for LLMs](https://mlsanity.com/paper/2503.22732/) (2025)
- [Generative AI Act II: Test Time Scaling Drives Cognition Engineering](https://mlsanity.com/paper/2504.13828/) (2025)
- [Co-Reinforcement Learning for Unified Multimodal Understanding and Generation](https://mlsanity.com/paper/2505.17534/) (2025)
- [Pre-Trained Policy Discriminators are General Reward Models](https://mlsanity.com/paper/2507.05197/) (2025)
- [CoIn: Counting the Invisible Reasoning Tokens in Commercial Opaque LLM APIs](https://mlsanity.com/paper/2505.13778/) (2025)

## Abstract (arXiv)

Large Language Models (LLMs) have transformed the natural language processing landscape and brought to life diverse applications. Pretraining on vast web-scale data has laid the foundation for these models, yet the research community is now increasingly shifting focus toward post-training techniques to achieve further breakthroughs. While pretraining provides a broad linguistic foundation, post-training methods enable LLMs to refine their knowledge, improve reasoning, enhance factual accuracy, and align more effectively with user intents and ethical considerations. Fine-tuning, reinforcement learning, and test-time scaling have emerged as critical strategies for optimizing LLMs performance, ensuring robustness, and improving adaptability across various real-world tasks. This survey provides a systematic exploration of post-training methodologies, analyzing their role in refining LLMs beyond pretraining, addressing key challenges such as catastrophic forgetting, reward hacking, and inference-time trade-offs. We highlight emerging directions in model alignment, scalable adaptation, and inference-time reasoning, and outline future research directions. We also provide a public repository to continually track developments in this fast-evolving field: https://github.com/mbzuai-oryx/Awesome-LLM-Post-training.