mlsanityData to 7 Oct 2026

Dynamic Important Example Mining for Reinforcement Finetuning

arXiv 2608.29252 ↗First posted 29 Aug 2026cs.AIcited by 0

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Reinforcement fine-tuning (RFT) is increasingly used to strengthen the reasoning abilities of large models, yet its effectiveness is bound by how training data are selected and used. Most data-centric RFT methods rely on static or heuristic sample selection, implicitly assuming a sample's value is fixed over training. This overlooks the non-stationary dynamics of policy learning and can lead to suboptimal updates. We propose Dynamic Important Example Mining (DIEM), a principled and fully automated framework that makes data utilization adaptive throughout RFT. DIEM integrates two components into each optimization step: (i) a gradient-alignment importance estimator that efficiently approximates each sample's marginal contribution to policy improvement; and (ii) a constrained batch reweighting scheme that maximizes aggregate utility while preserving the update's gradient magnitude to stabilize optimization. Across several reasoning benchmarks, DIEM consistently outperforms strong static and dynamic baselines. The code will be released via https://github.com/hrtan/DIEM.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 49 of its 56 references that are in the map. First posted on arXiv 29 Aug 2026.

Builds on

49 of 56 references.

Closest ancestors · 15

Same direction, newest first

  1. 2025Scaf-GRPO: Scaffolded Group Relative Policy Optimization for Enhancing LLM Reasoning22 Oct
  2. 2025SPEED-RL: Faster Training of Reasoning Models via Online Curriculum Learning10 Jun
  3. 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Example29 Apr
  4. 2025SRPO: A Cross-Domain Implementation of Large-Scale Reinforcement Learning on LLM19 Apr
  5. 2025VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Tasks7 Apr
  6. 2025GPG: A Simple and Strong Reinforcement Learning Baseline for Model Reasoning3 Apr
  7. 2025How Difficulty-Aware Staged Reinforcement Learning Enhances LLMs' Reasoning Capabilities: A Preliminary Experimental Study1 Apr
  8. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model31 Mar
  9. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  10. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild24 Mar
  11. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  12. 2025R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimization17 Mar
3 more
  1. 2025LIMR: Less is More for RL Scaling17 Feb
  2. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMs22 Jan
  3. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 34

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2024GPT-4o System Cardcited by 4,467
  4. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  5. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  6. 2015Trust Region Policy Optimizationcited by 2,503
  7. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  8. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  9. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  10. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  11. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  12. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 893
22 more
  1. 2016A Diagram Is Worth A Dozen Imagescited by 793
  2. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 669
  3. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 655
  4. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 653
  5. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 573
  6. 2020Estimating Training Data Influence by Tracing Gradient Descentcited by 565
  7. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 516
  8. 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalizationcited by 364
  9. 2019Selection via Proxy: Efficient Data Selection for Deep Learningcited by 337
  10. 2023Studying Large Language Model Generalization with Influence Functionscited by 311
  11. 2025MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learningcited by 294
  12. 2025SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Modelscited by 209
  13. 2019On the Accuracy of Influence Functions for Measuring Group Effectscited by 146
  14. 2024Improve Vision Language Model Chain-of-thought Reasoningcited by 135
  15. 2021Scaling Up Influence Functionscited by 130
  16. 2023DataInf: Efficiently Estimating Data Influence in LoRA-tuned LLMs and Diffusion Modelscited by 121
  17. 2025Skywork R1V: Pioneering Multimodal Reasoning with Chain-of-Thoughtcited by 69
  18. 2019Data Cleansing for Models Trained with SGDcited by 67
  19. 2025Prompt Curriculum Learning for Efficient LLM Post-Trainingcited by 40
  20. 2024Capturing the Temporal Dependence of Training Data Influencecited by 34
  21. 2022Quantile-Based Policy Optimization for Reinforcement Learningcited by 8
  22. 2025Understanding Data Influence with Differential Approximationcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Fine-Tuning on Self-Generated and Reward-Weighted Data: Learning Dynamics, Convergence Rates, and Benefits of Off-Policyness
  2. 2025Blending Supervised and Reinforcement Fine-Tuning with Prefix Sampling
  3. 2025Supervised Fine Tuning on Curated Data is Reinforcement Learning (and can be improved)
  4. 2026Finetuning with Sampling: SFT Learns Better Than You Think
  5. 2026JumpStart Your Policy Learning with Lessons from 160,000 Training Runs
  6. 2025On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectification
  7. 2025Mitigating Forgetting Between Supervised and Reinforcement Learning Yields Stronger Reasoners
  8. 2026Rethinking Importance Sampling in LLM Policy Optimization: A Cumulative Token Perspective