# Selective Self-to-Supervised Fine-Tuning for Generalization in Large Language Models

arXiv: https://arxiv.org/abs/2502.08130 · HTML: https://mlsanity.com/paper/2502.08130/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Feb 2025.

## Built on by (most cited first, 9 of 9)

- [Front-Loading Reasoning: The Synergy between Pretraining and Post-Training Data](https://mlsanity.com/paper/2510.03264/) (2025)
- [Reasoning Under 1 Billion: Memory-Augmented Reinforcement Learning for Large Language Models](https://mlsanity.com/paper/2504.02273/) (2025)
- [ExecVerify: White-Box RL with Verifiable Stepwise Rewards for Code Execution Reasoning](https://mlsanity.com/paper/2603.11226/) (2026)
- [Large Language Model Post-Training: A Unified View of Off-Policy and On-Policy Learning](https://mlsanity.com/paper/2604.07941/) (2026)
- [Learning from Mistakes: Negative Reasoning Samples Enhance Out-of-Domain Generalization](https://mlsanity.com/paper/2601.04992/) (2026)
- [CodeReasoner: Enhancing the Code Reasoning Ability with Reinforcement Learning](https://mlsanity.com/paper/2507.17548/) (2025)
- [DynaMiCS: Fine-tuning LLMs with Performance Constraints using Dynamic Mixtures](https://mlsanity.com/paper/2605.10770/) (2026)
- [LoRA-Edge: Tensor-Train-Assisted LoRA for Practical CNN Fine-Tuning on Edge Devices](https://mlsanity.com/paper/2511.03765/) (2025)
- [Match the Canvas, Not Just the Answer: Predictability-Aware SFT for Diffusion Language Models](https://mlsanity.com/paper/2610.18001/) (2026)

## Abstract (arXiv)

Fine-tuning Large Language Models (LLMs) on specific datasets is a common practice to improve performance on target tasks. However, this performance gain often leads to overfitting, where the model becomes too specialized in either the task or the characteristics of the training data, resulting in a loss of generalization. This paper introduces Selective Self-to-Supervised Fine-Tuning (S3FT), a fine-tuning approach that achieves better performance than the standard supervised fine-tuning (SFT) while improving generalization. S3FT leverages the existence of multiple valid responses to a query. By utilizing the model's correct responses, S3FT reduces model specialization during the fine-tuning stage. S3FT first identifies the correct model responses from the training set by deploying an appropriate judge. Then, it fine-tunes the model using the correct model responses and the gold response (or its paraphrase) for the remaining samples. The effectiveness of S3FT is demonstrated through experiments on mathematical reasoning, Python programming and reading comprehension tasks. The results show that standard SFT can lead to an average performance drop of up to 4.4 on multiple benchmarks, such as MMLU and TruthfulQA. In contrast, S3FT reduces this drop by half, i.e. 2.5, indicating better generalization capabilities than SFT while performing significantly better on the fine-tuning tasks.