# DeltaEvolve: Accelerating Scientific Discovery through Momentum-Driven Evolution

arXiv: https://arxiv.org/abs/2602.02919 · HTML: https://mlsanity.com/paper/2602.02919/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 30 references that are in the map.
- First posted on arXiv 2 Feb 2026.

## Builds on (most cited first, 22 of 22)

- [AlphaEvolve: A coding agent for scientific and algorithmic discovery](https://mlsanity.com/paper/2506.13131/) (2025)
- [The CMA Evolution Strategy: A Tutorial](https://mlsanity.com/paper/1604.00772/) (2016)
- [ShinkaEvolve: Towards Open-Ended And Sample-Efficient Program Evolution](https://mlsanity.com/paper/2509.19349/) (2025)
- [DreamCoder: Growing generalizable, interpretable knowledge with wake-sleep Bayesian program learning](https://mlsanity.com/paper/2006.08381/) (2020)
- [AutoML-Zero: Evolving Machine Learning Algorithms From Scratch](https://mlsanity.com/paper/2003.03384/) (2020)
- [Learning to Discover at Test Time](https://mlsanity.com/paper/2601.16175/) (2026)
- [Long Context Compression with Activation Beacon](https://mlsanity.com/paper/2401.03462/) (2024)
- [Neural Programmer-Interpreters](https://mlsanity.com/paper/1511.06279/) (2015)
- [MEMORYLLM: Towards Self-Updatable Large Language Models](https://mlsanity.com/paper/2402.04624/) (2024)
- [Mathematical exploration and discovery at scale](https://mlsanity.com/paper/2511.02864/) (2025)
- [LLM-SRBench: A New Benchmark for Scientific Equation Discovery with Large Language Models](https://mlsanity.com/paper/2504.10415/) (2025)
- [ThetaEvolve: Test-time Learning on Open Problems](https://mlsanity.com/paper/2511.23473/) (2025)
- [A Systematic Survey of Automatic Prompt Optimization Techniques](https://mlsanity.com/paper/2502.16923/) (2025)
- [M+: Extending MemoryLLM with Scalable Long-Term Memory](https://mlsanity.com/paper/2502.00592/) (2025)
- [CodePDE: An Inference Framework for LLM-driven PDE Solver Generation](https://mlsanity.com/paper/2505.08783/) (2025)
- [AlgoTune: Can Language Models Speed Up General-Purpose Numerical Programs?](https://mlsanity.com/paper/2507.15887/) (2025)
- [LIDDIA: Language-based Intelligent Drug Discovery Agent](https://mlsanity.com/paper/2502.13959/) (2025)
- [Let's (not) just put things in Context: Test-Time Training for Long-Context LLMs](https://mlsanity.com/paper/2512.13898/) (2025)
- [Efficient Prompt Compression with Evaluator Heads for Long-Context Transformer Inference](https://mlsanity.com/paper/2501.12959/) (2025)
- [GigaEvo: An Open Source Optimization Framework Powered By LLMs And Evolution Algorithms](https://mlsanity.com/paper/2511.17592/) (2025)
- [No Loss, No Gain: Gated Refinement and Adaptive Compression for Prompt Optimization](https://mlsanity.com/paper/2509.23387/) (2025)
- [Compositional Generalization and Decomposition in Neural Program Synthesis](https://mlsanity.com/paper/2204.03758/) (2022)

## Built on by (most cited first, 26 of 26)

- [AdaEvolve: Adaptive LLM Driven Zeroth-Order Optimization](https://mlsanity.com/paper/2602.20133/) (2026)
- [EvoX: Meta-Evolution for Automated Discovery](https://mlsanity.com/paper/2602.23413/) (2026)
- [Dream-RSI: Recursive Self-Improvement through Evolving Worlds](https://mlsanity.com/paper/2609.14858/) (2026)
- [NatureBench: Can Coding Agents Match the Published SOTA of Nature-Family Papers?](https://mlsanity.com/paper/2606.24530/) (2026)
- [Automated Discovery Has No Universally Superior Harness](https://mlsanity.com/paper/2607.18235/) (2026)
- [The Agentic Researcher: A Practical Guide to AI-Assisted Research in Mathematics and Machine Learning](https://mlsanity.com/paper/2603.15914/) (2026)
- [BenchEvolver: Frontier Task Synthesis via Solution-Centric Evolution](https://mlsanity.com/paper/2606.01286/) (2026)
- [Evolutionary Multi-Task Optimization for LLM-Guided Program Discovery](https://mlsanity.com/paper/2605.22613/) (2026)
- [SMCEvolve: Principled Scientific Discovery via Sequential Monte Carlo Evolution](https://mlsanity.com/paper/2605.15308/) (2026)
- [Socratic agents for autonomous scientific discovery in high-dimensional physical systems](https://mlsanity.com/paper/2606.26722/) (2026)
- [What is Missing from AI Post-Training AI: An Empirical Analysis](https://mlsanity.com/paper/2608.19072/) (2026)
- [Marginal Advantage Accumulation for Memory-Driven Agent Self-Evolution](https://mlsanity.com/paper/2606.20475/) (2026)
- [From Solvers to Research: Large Language Model-Driven Formal Mathematics at the Research Frontier](https://mlsanity.com/paper/2607.07779/) (2026)
- [Teaching LLMs to Self-Evolve: Cultivating Core Meta-Skills with Reinforcement Learning](https://mlsanity.com/paper/2607.21971/) (2026)
- [The Optimizer Is the Agent: Reasoning-Driven Search across Prompts, Programs, and ML Workflows](https://mlsanity.com/paper/2608.06714/) (2026)
- [k-server-bench: Automating Potential Discovery for the k-Server Conjecture](https://mlsanity.com/paper/2604.07240/) (2026)
- [Co-evolving Agent Architectures and Interpretable Reasoning for Automated Optimization](https://mlsanity.com/paper/2604.17708/) (2026)
- [Budget-Aware LLM Discovery via Cost-Calibrated Frontier Utility](https://mlsanity.com/paper/2607.26828/) (2026)
- [Hill Sampling for Test-Time Scaling: A Simple and Better Alternative to Repeated Sampling, Evolution, and Training](https://mlsanity.com/paper/2609.25510/) (2026)
- [Collaborative Principle Evolution via Evidence Transfer for Scientific Discovery](https://mlsanity.com/paper/2609.35315/) (2026)
- [HeurEvo: Agentic Evolution of Hybrid Solver-Augmented Heuristics for Time-Critical Mathematical Optimization](https://mlsanity.com/paper/2609.36303/) (2026)
- [Component-Aware Feedback for Self-Evolving Programs](https://mlsanity.com/paper/2609.38639/) (2026)
- [LabBook: Harnessing Experimental History for Efficient LLM-Driven Discovery](https://mlsanity.com/paper/2610.00675/) (2026)
- [SEDIMA: Cross-Run Hierarchical Insight Memory for Evolutionary Search Agents](https://mlsanity.com/paper/2610.02361/) (2026)
- [DuoDiscover: Scaling Autonomous Discovery Beyond the Evaluation Bottleneck](https://mlsanity.com/paper/2610.17184/) (2026)
- [SHELDON: Reasoning through Dual Search over Hypothesis and Observation Spaces](https://mlsanity.com/paper/2610.18025/) (2026)

## Abstract (arXiv)

LLM-driven evolutionary systems have shown promise for automated science discovery, yet existing approaches such as AlphaEvolve rely on full-code histories that are context-inefficient and potentially provide weak evolutionary guidance. In this work, we first formalize the evolutionary agents as a general Expectation-Maximization framework, where the language model samples candidate programs (E-step) and the system updates the control context based on evaluation feedback (M-step). Under this view, constructing context via full-code snapshots constitutes a suboptimal M-step, as redundant implement details dilutes core algorithmic ideas, making it difficult to provide clear inspirations for evolution. To address this, we propose DeltaEvolve, a momentum-driven evolutionary framework that replaces full-code history with structured semantic delta capturing how and why modifications between successive nodes affect performance. As programs are often decomposable, semantic delta usually contains many effective components which are transferable and more informative to drive improvement. By organizing semantic delta through multi-level database and progressive disclosure mechanism, input tokens are further reduced. Empirical evaluations on tasks across diverse scientific domains show that our framework can discover better solution with less token consumption over full-code-based evolutionary agents.