mlsanityData to 7 Oct 2026

Rotation-Preserving Supervised Fine-Tuning

arXiv 2605.10973 ↗First posted 8 May 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Supervised fine-tuning (SFT) improves in-domain performance but can degrade out-of-domain (OOD) generalization. Prior work suggests that this degradation is related to changes in dominant singular subspaces of pretrained weight matrices. However, directly identifying loss-sensitive directions with Hessian or Fisher information is computationally expensive at LLM scale. In this work, we propose preserving projected rotations in pretrained singular subspaces as an efficient proxy for Fisher-sensitive directions, which we call Rotation-Preserving Supervised Fine-Tuning (RPSFT). RPSFT penalizes changes in the projected top-k singular-vector block of each pretrained weight matrix, limiting unnecessary rotation while preserving task adaptation. Across model families and sizes trained on math reasoning data, RPSFT improves the in-domain/OOD trade-off over standard SFT and strong SFT baselines, better preserves pretrained representations, and provides stronger initializations for downstream RL fine-tuning. Code is available at hrefhttps://github.com/jinhangzhan/RPSFT.githttps://github.com/jinhangzhan/RPSFT.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 40 of its 53 references that are in the map. First posted on arXiv 8 May 2026.

Builds on

40 of 53 references.

References · 40

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  6. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  7. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  8. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  9. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  10. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  11. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  12. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
28 more
  1. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  2. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  3. 2014New insights and perspectives on the natural gradient methodcited by 435
  4. 2023Orthogonal Subspace Learning for Language Model Continual Learningcited by 296
  5. 2024PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Modelscited by 261
  6. 2019Limitations of the Empirical Fisher Approximation for Natural Gradient Descentcited by 213
  7. 2025RL's Razor: Why Online Reinforcement Learning Forgets Lesscited by 162
  8. 2025On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectificationcited by 121
  9. 2025Does Math Reasoning Improve General LLM Capabilities? Understanding Transferability of LLM Reasoningcited by 101
  10. 2024MiLoRA: Harnessing Minor Singular Components for Parameter-Efficient LLM Finetuningcited by 89
  11. 2023Maintaining Plasticity in Continual Learning via Regenerative Regularizationcited by 63
  12. 2025The Path Not Taken: RLVR Provably Learns Off the Principalscited by 52
  13. 2025Reinforcement Fine-Tuning Naturally Mitigates Forgetting in Continual Post-Trainingcited by 42
  14. 2025Supervised Fine Tuning on Curated Data is Reinforcement Learning (and can be improved)cited by 37
  15. 2024Learning Continually by Spectral Regularizationcited by 34
  16. 2024Weight Clipping for Deep Continual and Reinforcement Learningcited by 29
  17. 2025RL Fine-Tuning Heals OOD Forgetting in SFTcited by 27
  18. 2025SFT Doesn't Always Hurt General Capabilities: Revisiting Domain-Specific Fine-Tuning in LLMscited by 23
  19. 2024What Do Learning Dynamics Reveal About Generalization in LLM Reasoning?cited by 22
  20. 2024Parseval Regularization for Continual Reinforcement Learningcited by 21
  21. 2025Learning to Reason at the Frontier of Learnabilitycited by 20
  22. 2024Understanding Layer Significance in LLM Alignmentcited by 19
  23. 2025Sculpting Subspaces: Constrained Full Fine-Tuning in LLMs for Continual Learningcited by 15
  24. 2025Rethinking the Stability-Plasticity Trade-off in Continual Learning from an Architectural Perspectivecited by 14
  25. 2025Continual Gradient Low-Rank Projection Fine-Tuning for LLMscited by 10
  26. 2025Offline Learning and Forgetting for Reasoning with Large Language Modelscited by 4
  27. 2023Hessian Eigenvectors and Principal Component Analysis of Neural Network Weight Matricescited by 3
  28. 2026A Layer-wise Analysis of Supervised Fine-Tuningcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Anchored Supervised Fine-Tuning
  2. 2024PACE: Marrying generalization in PArameter-efficient fine-tuning with Consistency rEgularization
  3. 2026LOFT: Low-Rank Orthogonal Fine-Tuning via Task-Aware Support Selection
  4. 2025Singular Value Decomposition on Kronecker Adaptation for Large Language Model
  5. 2026CORA: Per-Slice Coherent Orthogonal Rotation for SVD-based Low-Rank Adaptation
  6. 2025HyperAdapt: Simple High-Rank Adaptation
  7. 2026PriFT: Prior-Support Guided Supervised Fine-Tuning
  8. 2025LIFT the Veil for the Truth: Principal Weights Emerge after Rank Reduction for Reasoning-Focused Supervised Fine-Tuning