mlsanityData to 14 Oct 2026

Beyond Distribution Matching: Selective Margin Preservation for Supervised Fine-Tuning

arXiv 2610.19685 ↗First posted 14 Oct 2026cs.LG cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Supervised fine-tuning (SFT) changes logit margins among non-target tokens at teacher-forced prefixes. Matching a local reference distribution keeps those probabilities and also penalizes margin expansions. We ask whether a soft auxiliary can preserve selected local non-target margin relations without freezing that distribution. We introduce Selective Rank-Preserving SFT (SRP-SFT), a soft margin target: a hinge on reference-scaled lower bounds for pairs from the reference's top-K non-target tokens, applied at fixed teacher-forced prefixes. The hinge stops once the requested bound is met. We derive the KL cost of violating a margin, characterize the feasible region and its conditional utility, and give a finite-step condition for reducing aggregate violations. In one-epoch experiments on Qwen3-4B-Base and UltraFeedback, this target raises MATH-500 and code-generation scores over CE and LP-SFT at all three training seeds. At seed 1234, MATH-500 pass@1 rises from 49.56% to 54.56% and the four-metric code mean rises from 65.91% to 69.50% relative to LP-SFT. Ranking controls, held-out cache diagnostics, sensitivity sweeps, and six additional backbones show where these score changes appear.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 34 of its 52 references that are in the map. First posted on arXiv 14 Oct 2026.

Builds on

34 of 52 references.

References · 34

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  6. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  8. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  9. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  10. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  11. 2021Program Synthesis with Large Language Modelscited by 3,637
  12. 2021Convex optimizationcited by 2,515
22 more
  1. 2016Learning without Forgettingcited by 2,496
  2. 2017Gradient Episodic Memory for Continual Learningcited by 1,898
  3. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  4. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
  5. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
  6. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 939
  7. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 639
  8. 2023UltraFeedback: Boosting Language Models with Scaled AI Feedbackcited by 406
  9. 2018Explicit Inductive Bias for Transfer Learning with Convolutional Networkscited by 227
  10. 2022Knowledge Distillation from A Stronger Teachercited by 208
  11. 2017Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulationcited by 185
  12. 2025AIMO-2 Winning Solution: Building State-of-the-Art Mathematical Reasoning Models with OpenMathReasoning datasetcited by 174
  13. 2020Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillationcited by 145
  14. 2020A Theoretical Analysis of Catastrophic Forgetting through the NTK Overlap Matrixcited by 103
  15. 2025rStar-Coder: Scaling Competitive Code Reasoning with a Large-Scale Verified Datasetcited by 51
  16. 2024BiLD: Bi-directional Logits Difference Loss for Large Language Model Distillationcited by 19
  17. 2026Entropy-Adaptive Fine-Tuning: Resolving Confident Conflicts to Mitigate Forgettingcited by 16
  18. 2024Enhancing Knowledge Distillation of Large Language Models through Efficient Multi-Modal Distribution Alignmentcited by 11
  19. 2024Enhancing Logits Distillation with Plug&Play Kendall's τ Ranking Losscited by 3
  20. 2025Learning from the Undesirable: Robust Adaptation of Language Models without Forgettingcited by 3
  21. 2026Rotation-Preserving Supervised Fine-Tuningcited by 3
  22. 2026LP-SFT: Keeping Plausible Alternatives Alive in Supervised Fine-Tuningcited by 1

Built on by

Not cited yet.