mlsanityData to 7 Oct 2026

Supervised Fine-Tuning versus Reinforcement Learning: A Study of Post-Training Methods for Large Language Models

arXiv 2603.13985 ↗First posted 14 Mar 2026cs.AI cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

Pre-trained Large Language Model (LLM) exhibits broad capabilities, yet, for specific tasks or domains their attainment of higher accuracy and more reliable reasoning generally depends on post-training through Supervised Fine-Tuning (SFT) or Reinforcement Learning (RL). Although often treated as distinct methodologies, recent theoretical and empirical developments demonstrate that SFT and RL are closely connected. This study presents a comprehensive and unified perspective on LLM post-training with SFT and RL. We first provide an in-depth overview of both techniques, examining their objectives, algorithmic structures, and data requirements. We then systematically analyze their interplay, highlighting frameworks that integrate SFT and RL, hybrid training pipelines, and methods that leverage their complementary strengths. Drawing on a representative set of recent application studies from 2023 to 2025, we identify emerging trends, characterize the rapid shift toward hybrid post-training paradigms, and distill key takeaways that clarify when and why each method is most effective. By synthesizing theoretical insights, practical methodologies, and empirical evidence, this study establishes a coherent understanding of SFT and RL within a unified framework and outlines promising directions for future research in scalable, efficient, and generalizable LLM post-training.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 122 of its 147 references that are in the map. First posted on arXiv 14 Mar 2026.

Builds on

122 of 147 references.

References · 122

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  7. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  8. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  9. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  10. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  11. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  12. 2023LIMA: Less Is More for Alignmentcited by 1,080
38 more
  1. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 895
  2. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 728
  3. 2022Large Language Models Can Self-Improvecited by 656
  4. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 655
  5. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 618
  6. 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 450
  7. 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 442
  8. 2023Llemma: An Open Language Model For Mathematicscited by 385
  9. 2020Generative Language Modeling for Automated Theorem Provingcited by 379
  10. 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 308
  11. 2019Causal Confusion in Imitation Learningcited by 286
  12. 2025Reasoning with Exploration: An Entropy Perspectivecited by 277
  13. 2025Learning to Reason under Off-Policy Guidancecited by 266
  14. 2025SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolutioncited by 246
  15. 2024WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learningcited by 243
  16. 2024ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RLcited by 212
  17. 2024Trial and Error: Exploration-Based Trajectory Optimization for LLM Agentscited by 194
  18. 2024OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Datacited by 194
  19. 2025The Surprising Effectiveness of Negative Reinforcement in LLM Reasoningcited by 158
  20. 2023MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoningcited by 155
  21. 2024OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Datasetcited by 143
  22. 2024AutoWebGLM: A Large Language Model-based Web Navigating Agentcited by 130
  23. 2025On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectificationcited by 121
  24. 2024AgentGym: Evolving Large Language Model-based Agents across Diverse Environmentscited by 120
  25. 2023Retroformer: Retrospective Large Language Agents with Policy Gradient Optimizationcited by 110
  26. 2024Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizercited by 109
  27. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 108
  28. 2025Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rolloutscited by 108
  29. 2025Self-Evolving Curriculum for LLM Reasoningcited by 107
  30. 2024SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationalescited by 102
  31. 2025On-Policy RL Meets Off-Policy Experts: Harmonizing Supervised Fine-Tuning and Reinforcement Learning via Dynamic Weightingcited by 88
  32. 2025SEED-GRPO: Semantic Entropy Enhanced GRPO for Uncertainty-Aware Policy Optimizationcited by 87
  33. 2024Watch Every Step! LLM Agent Learning via Iterative Step-Level Process Refinementcited by 86
  34. 2024Unfamiliar Finetuning Examples Control How Language Models Hallucinatecited by 85
  35. 2025Not All Rollouts are Useful: Down-Sampling Rollouts in LLM Reinforcement Learningcited by 84
  36. 2025PlanGenLLMs: A Modern Survey of LLM Planning Capabilitiescited by 82
  37. 2025SRFT: A Single-Stage Method with Supervised and Reinforcement Fine-Tuning for Reasoningcited by 80
  38. 2025Sample More to Think Less: Group Filtered Policy Optimization for Concise Reasoningcited by 75

50 most cited of 122 All 122 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.