mlsanityData to 7 Oct 2026

RIFT: Repurposing Negative Samples via Reward-Informed Fine-Tuning

arXiv 2601.09253 ↗First posted 14 Jan 2026cs.LG cs.AIcited by 3

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

also in 1 more
Abstract (arXiv)

While Supervised Fine-Tuning (SFT) and Rejection Sampling Fine-Tuning (RFT) are standard for LLM alignment, they either rely on costly expert data or discard valuable negative samples, leading to data inefficiency. To address this, we propose Reward Informed Fine-Tuning (RIFT), a simple yet effective framework that utilizes all self-generated samples. Unlike the hard thresholding of RFT, RIFT repurposes negative trajectories, reweighting the loss with scalar rewards to learn from both the positive and negative trajectories from the model outputs. To overcome the training collapse caused by naive reward integration, where direct multiplication yields an unbounded loss, we introduce a stabilized loss formulation that ensures numerical robustness and optimization efficiency. Extensive experiments on mathematical benchmarks across various base models show that RIFT consistently outperforms RFT. Our results demonstrate that RIFT is a robust and data-efficient alternative for alignment using mixed-quality, self-generated data.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 24 of its 44 references that are in the map. First posted on arXiv 14 Jan 2026.

Builds on

24 of 44 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024SimPO: Simple Preference Optimization with a Reference-Free Reward23 May
  2. 2023LIMA: Less Is More for Alignment18 May
  3. 2022Training language models to follow instructions with human feedback4 Mar

Other references · 21

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  5. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  6. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  7. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  8. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  9. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 655
  10. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 622
  11. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 552
  12. 2020Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stoppingcited by 377
9 more
  1. 2024SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuningcited by 345
  2. 2025GPG: A Simple and Strong Reinforcement Learning Baseline for Model Reasoningcited by 147
  3. 2025On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectificationcited by 121
  4. 2022On Reinforcement Learning and Distribution Matching for Fine-Tuning Language Models with no Catastrophic Forgettingcited by 80
  5. 2024OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AIcited by 70
  6. 2025Stabilizing Reinforcement Learning with LLMs: Formulation and Practicescited by 38
  7. 2025Supervised Fine Tuning on Curated Data is Reinforcement Learning (and can be improved)cited by 37
  8. 2025Beyond Log Likelihood: Probability-Based Objectives for Supervised Fine-Tuning across the Model Capability Continuumcited by 16
  9. 2025Tiny Model, Big Logic: Diversity-Driven Optimization Elicits Large-Model Reasoning Ability in VibeThinker-1.5Bcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026ASymPO: Asymmetric-Scale Policy Optimization for Asynchronous LLM Post-Training Without Behavior Informationcited by 2
  2. 2026Weight-Space Geometry of Offline Reasoning Trainingcited by 0
  3. 2026ROSS: Relearning from Self-Generated Rollouts through Selective Supervisioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024As Simple as Fine-tuning: LLM Alignment via Bidirectional Negative Feedback Loss
  2. 2025Layer-Aware Representation Filtering: Purifying Finetuning Data to Preserve LLM Safety Alignment
  3. 2024UNA: A Unified Supervised Framework for Efficient LLM Alignment Across Feedback Types
  4. 2026ExpBoN: Exponential-Noise Best-of-n for Efficient Test-Time LLM Alignment
  5. 2025The Differences Between Direct Alignment Algorithms are a Blur
  6. 2026Why Is RLHF Alignment Shallow? A Gradient Analysis
  7. 2025Vulnerability-Aware Alignment: Mitigating Uneven Forgetting in Harmful Fine-Tuning
  8. 2024H3Fusion: Helpful, Harmless, Honest Fusion of Aligned LLMs