mlsanityData to 14 Oct 2026

DynamixSFT: Dynamic Mixture Optimization of Instruction Tuning Collections

arXiv 2508.12116 ↗First posted 16 Aug 2025cs.LG cs.AI cs.CLcited by 9

Not on the map. Why?

Abstract (arXiv)

As numerous instruction-tuning datasets continue to emerge, dynamically balancing and optimizing their mixtures has become a critical challenge. To address this, we propose DynamixSFT, a dynamic and automated method for instruction-tuning dataset mixture optimization. We formulate the problem as a multi-armed bandit setup and introduce a Prior-scaled Boltzmann Exploration that softly anchors the updated sampling distribution to the original dataset proportions, thereby preserving the inherent diversity and coverage of the collection. Sampling probabilities are updated using a lightweight 1-Step Look-ahead Reward, reflecting how much the dataset contributes to improving the model's performance at its current state. We demonstrate that DynamixSFT effectively optimizes the Tulu-2-mixture and Tulu-3-mixture collections across 10 benchmarks, while introducing minimal computational overhead over naive sampling. Furthermore, we provide a comprehensive analysis and visualizations to offer deeper insights into the adaptive dynamics of our method.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 34 of its 42 references that are in the map. First posted on arXiv 16 Aug 2025.

Builds on

34 of 42 references.

References · 34

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  4. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  5. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  6. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  7. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
  8. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
  9. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,125
  10. 2023LIMA: Less Is More for Alignmentcited by 1,100
  11. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
  12. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 903
22 more
  1. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 739
  2. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 733
  3. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 721
  4. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 629
  5. 2020Fact or Fiction: Verifying Scientific Claimscited by 554
  6. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 486
  7. 2023How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resourcescited by 408
  8. 2023DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretrainingcited by 399
  9. 2018Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Constructioncited by 322
  10. 2024OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Datacited by 200
  11. 2024RegMix: Data Mixture as Regression for Language Model Pre-trainingcited by 177
  12. 2020TLDR: Extreme Summarization of Scientific Documentscited by 136
  13. 2025Self-Evolving Curriculum for LLM Reasoningcited by 112
  14. 2024The Art of Saying No: Contextual Noncompliance in Language Modelscited by 105
  15. 2023DoGE: Domain Reweighting with Generalization Estimationcited by 90
  16. 2023Efficient Online Data Mixing For Language Model Pre-Trainingcited by 77
  17. 2020Balancing Training for Multilingual Neural Machine Translationcited by 72
  18. 2017Boltzmann Exploration Done Rightcited by 69
  19. 2019Inferring Which Medical Treatments Work from Reports of Clinical Trialscited by 64
  20. 2021Uncertainty-Aware Balancing for Multilingual and Multi-Domain Neural Machine Translation Trainingcited by 19
  21. 2024Dynamic Data Mixing Maximizes Instruction Tuning for Mixture-of-Expertscited by 19
  22. 2019Almost Boltzmann Explorationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 9.

9 papers.

  1. 2026DynaMiCS: Fine-tuning LLMs with Performance Constraints using Dynamic Mixturescited by 4
  2. 2026mSFT: Addressing Dataset Mixtures Overfitting Heterogeneously in Multi-task SFTcited by 3
  3. 2026A Component-Based Survey of Interactions between Large Language Models and Multi-Armed Banditscited by 1
  4. 2026Beyond Full-Model Rollback: AuroSFT for Adapter-State Multi-Task Fine-Tuningcited by 0
  5. 2026How code helps different tasks? A decompositional lens on LLM post-trainingcited by 0
  6. 2026CALIBUDGET: Calibration-Guided Source Allocation for Fixed-Budget Mixed-Reasoning Adaptationcited by 0
  7. 2026Adaptive Mutual Distillation for Balanced Multi-Task Post-Training of Large Language Modelscited by 0
  8. 2026Dynamic Minimax Regret Optimization for Robust LLM Post-Trainingcited by 0
  9. 2026LLM Agents for Automated Post-Training via Structured Recipe Optimizationcited by 0