# DIVE: Unlocking Self-Improvement in Frozen Language Models Through Diversity-Driven Skill Evolution

arXiv: https://arxiv.org/abs/2608.12486 · HTML: https://mlsanity.com/paper/2608.12486/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Self-rewarding alignment and self-training, one of 13 directions in Synthetic data and self-improvement for LLMs.
- Builds on 33 of its 63 references that are in the map.
- First posted on arXiv 12 Aug 2026.

## Placed in

- Synthetic data and self-improvement for LLMs / [Self-rewarding alignment and self-training](https://mlsanity.com/synth/self-rewarding-alignment-and-self-training/)

## Builds on (most cited first, 30 of 33)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [HybridFlow: A Flexible and Efficient RLHF Framework](https://mlsanity.com/paper/2409.19256/) (2024)
- [DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence](https://mlsanity.com/paper/2606.19348/) (2026)
- [OpenAI GPT-5 System Card](https://mlsanity.com/paper/2601.03267/) (2025)
- [Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution](https://mlsanity.com/paper/2309.16797/) (2023)
- [GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning](https://mlsanity.com/paper/2507.19457/) (2025)
- [Reinforced Self-Training (ReST) for Language Modeling](https://mlsanity.com/paper/2308.08998/) (2023)
- [Agent Workflow Memory](https://mlsanity.com/paper/2409.07429/) (2024)
- [Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models](https://mlsanity.com/paper/2312.06585/) (2023)
- [ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory](https://mlsanity.com/paper/2509.25140/) (2025)
- [Qwen3.5-Omni Technical Report](https://mlsanity.com/paper/2604.15804/) (2026)
- [SkillOpt: Executive Strategy for Self-Evolving Agent Skills](https://mlsanity.com/paper/2605.23904/) (2026)
- [Beyond Benchmarks: MathArena as an Evaluation Platform for Mathematics with LLMs](https://mlsanity.com/paper/2605.00674/) (2026)
- [AutoSkill: Experience-Driven Lifelong Learning via Skill Self-Evolution](https://mlsanity.com/paper/2603.01145/) (2026)
- [Inducing Programmatic Skills for Agentic Tasks](https://mlsanity.com/paper/2504.06821/) (2025)
- [PromptWizard: Task-Aware Prompt Optimization Framework](https://mlsanity.com/paper/2405.18369/) (2024)
- [AgentRouter: A Knowledge-Graph-Guided LLM Router for Collaborative Multi-Agent Question Answering](https://mlsanity.com/paper/2510.05445/) (2025)
- [LLMs Are In-Context Bandit Reinforcement Learners](https://mlsanity.com/paper/2410.05362/) (2024)
- [Formalizing Learning from Language Feedback with Provable Guarantees](https://mlsanity.com/paper/2506.10341/) (2025)
- [Enhancing Long Chain-of-Thought Reasoning through Multi-Path Plan Aggregation](https://mlsanity.com/paper/2510.11620/) (2025)
- [Adaptive Information Control for Search-Augmented LLM Reasoning](https://mlsanity.com/paper/2602.01672/) (2026)
- [The Equational Theories Project: Advancing Collaborative Mathematical Research at Scale](https://mlsanity.com/paper/2512.07087/) (2025)
- [Long-Context Modeling with Dynamic Hierarchical Sparse Attention for Memory-Constrained LLM Inference](https://mlsanity.com/paper/2510.24606/) (2025)
- [PathWise: Planning through World Model for Automated Heuristic Design via Self-Evolving LLMs](https://mlsanity.com/paper/2601.20539/) (2026)
- [Natural Language Reinforcement Learning](https://mlsanity.com/paper/2411.14251/) (2024)
- [Conflict-Aware Fusion: Mitigating Logic Inertia in Large Language Models via Structured Cognitive Priors](https://mlsanity.com/paper/2512.06393/) (2025)
- [Optimizing What Policies Learn From: Recoverability-aware Rollout Intervention Learning](https://mlsanity.com/paper/2608.05080/) (2026)

## Built on by (most cited first, 5 of 5)

- [Strategically Diverse Sampling for Self-Training](https://mlsanity.com/paper/2609.31571/) (2026)
- [AdaGEPA: Adaptive Feedback Allocation for Reflective Prompt Optimization](https://mlsanity.com/paper/2609.39927/) (2026)
- [Harnessing LLMs as Agents: What Does It Cost?](https://mlsanity.com/paper/2610.02488/) (2026)
- [CLEAT: Learning Value-Guided Clarification for Interactive Agents](https://mlsanity.com/paper/2610.12655/) (2026)
- [From Existing Methods to New Research Ideas: Two Perspectives on Recursive Self-Improvement](https://mlsanity.com/paper/2610.14625/) (2026)

## Abstract (arXiv)

Large language models (LLMs) cannot retain post-deployment experience without parameter updates. We introduce DIVE, a diversity-driven framework that enables frozen LLMs to improve by evolving persistent natural-language skills from task experience and verifier feedback. These skills encode reusable reasoning procedures, verification strategies, common failure modes, and output constraints and are both executed and revised by the same underlying model without access to a teacher model. Since natural-language skill evolution is a stochastic, non-convex search process, optimizing a single skill trajectory can overfit to sampled experience or converge to a suboptimal solution. DIVE mitigates this optimization variance by independently evolving multiple skill populations from bootstrapped experience, adaptively refining them through diverse transformations, and jointly selecting a complementary set of skills. Across six mathematical and logical reasoning tasks and multiple model families, DIVE consistently outperforms existing reasoning methods, prompt-optimization approaches, skill-development frameworks, and memory-based baselines. It achieves rapid self-improvement from accumulated experience, obtaining substantially larger performance gains with fewer rollouts than parameter-based methods such as SFT and GRPO, and prompt optimization with GEPA. Further, the resulting skills transfer across model scales and families, enabling smaller models such as GPT-5-nano to match or outperform larger counterparts, i.e., GPT-5, under conventional prompting. These results establish diversity-driven skill evolution as an effective, interpretable, and parameter-free approach to LLM self-improvement.