mlsanityData to 7 Oct 2026

DIVE: Diversified Iterative Self-Improvement

arXiv 2501.00747 ↗First posted 1 Jan 2025cs.CLcited by 3

On the map: Synthetic data and self-improvement for LLMs → Self-rewarding alignment and self-training

Abstract (arXiv)

Recent advances in large language models (LLMs) have demonstrated the effectiveness of Iterative Self-Improvement (ISI) techniques. However, continuous training on self-generated data leads to reduced output diversity, a limitation particularly critical in reasoning tasks where diverse solution paths are essential. We present DIVE (Diversified Iterative Self-Improvement), a novel framework that addresses this challenge through two key components: Sample Pool Expansion for broader solution exploration, and Data Selection for balancing diversity and quality in preference pairs. Experiments on MATH and GSM8k datasets show that DIVE achieves a 10% to 45% relative increase in output diversity metrics while maintaining performance quality compared to vanilla ISI. Our ablation studies confirm both components' significance in achieving these improvements. Code is available at https://github.com/qinyiwei/DIVE.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Self-rewarding alignment and self-training, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 27 of its 42 references that are in the map. First posted on arXiv 1 Jan 2025.

Builds on

27 of 42 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Progress or Regress? Self-Improvement Reversal in Post-training6 Jul

Other references · 26

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  7. 2023Mistral 7Bcited by 3,065
  8. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  9. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,068
  10. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  11. 2022MTEB: Massive Text Embedding Benchmarkcited by 623
  12. 2023SLiC-HF: Sequence Likelihood Calibration with Human Feedbackcited by 395
14 more
  1. 2023Understanding the Effects of RLHF on LLM Generalisation and Diversitycited by 386
  2. 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 343
  3. 2024KTO: Model Alignment as Prospect Theoretic Optimizationcited by 256
  4. 2024Iterative Reasoning Preference Optimizationcited by 244
  5. 2022Making Large Language Models Better Reasoners with Step-Aware Verifiercited by 231
  6. 2024Orca-Math: Unlocking the potential of SLMs in Grade School Mathcited by 151
  7. 2024Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Datacited by 143
  8. 2020Evaluating the Evaluation of Diversity in Natural Language Generationcited by 127
  9. 2024A Tale of Tails: Model Collapse as a Change of Scaling Lawscited by 127
  10. 2023Some things are more CRINGE than others: Iterative Preference Optimization with the Pairwise Cringe Losscited by 108
  11. 2024NV-Retriever: Improving text embedding models with effective hard-negative miningcited by 83
  12. 2016A Simple, Fast Diverse Decoding Algorithm for Neural Generationcited by 58
  13. 2024ChatGLM-Math: Improving Math Problem-Solving in Large Language Models with a Self-Critique Pipelinecited by 47
  14. 2024Flow of Reasoning: Training LLMs for Divergent Reasoning with Minimal Examplescited by 29

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Diverse Preference Optimizationcited by 48
  2. 2026Self-Improvements in Modern Agentic Systems: A Surveycited by 17
  3. 2026Unlocking LLM Creativity in Science through Analogical Reasoningcited by 6

Similar, not cited

Close in wording, no citation either way

  1. 2026DIVE: Unlocking Self-Improvement in Frozen Language Models Through Diversity-Driven Skill Evolution
  2. 2026Better, Faster: Harnessing Self-Improvement in Large Reasoning Models
  3. 2024Mitigating Tail Narrowing in LLM Self-Improvement via Socratic-Guided Sampling
  4. 2026A Task-Centric Theory for Iterative Self-Improvement with Easy-to-Hard Curricula
  5. 2025Theoretical Modeling of Large Language Model Self-Improvement Training Dynamics Through Solver-Verifier Gap
  6. 2024Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Models
  7. 2025Iterative Deepening Sampling as Efficient Test-Time Scaling
  8. 2026Self-Improvement of Large Language Models: A Technical Overview and Future Outlook