# MLEvolve: A Self-Evolving Framework for Automated Machine Learning Algorithm Discovery

arXiv: https://arxiv.org/abs/2606.06473 · HTML: https://mlsanity.com/paper/2606.06473/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Placed in Self-evolving LLM agents, one of 13 directions in Synthetic data and self-improvement for LLMs.
- Builds on 32 of its 39 references that are in the map.
- First posted on arXiv 4 Jun 2026.

## Placed in

- Synthetic data and self-improvement for LLMs / [Self-evolving LLM agents](https://mlsanity.com/synth/self-evolving-llm-agents/)

## Builds on (most cited first, 30 of 32)

- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [A-MEM: Agentic Memory for LLM Agents](https://mlsanity.com/paper/2502.12110/) (2025)
- [OpenHands: An Open Platform for AI Software Developers as Generalist Agents](https://mlsanity.com/paper/2407.16741/) (2024)
- [AlphaEvolve: A coding agent for scientific and algorithmic discovery](https://mlsanity.com/paper/2506.13131/) (2025)
- [A Survey on the Memory Mechanism of Large Language Model based Agents](https://mlsanity.com/paper/2404.13501/) (2024)
- [MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering](https://mlsanity.com/paper/2410.07095/) (2024)
- [AutoML: A Survey of the State-of-the-Art](https://mlsanity.com/paper/1908.00709/) (2019)
- [MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation](https://mlsanity.com/paper/2310.03302/) (2023)
- [AIDE: AI-Driven Exploration in the Space of Code](https://mlsanity.com/paper/2502.13138/) (2025)
- [Auto-Sklearn 2.0: Hands-free AutoML via Meta-Learning](https://mlsanity.com/paper/2007.04074/) (2020)
- [Towards End-to-End Automation of AI Research](https://mlsanity.com/paper/2606.15497/) (2026)
- [Learning to Discover at Test Time](https://mlsanity.com/paper/2601.16175/) (2026)
- [LocAgent: Graph-Guided LLM Agents for Code Localization](https://mlsanity.com/paper/2503.09089/) (2025)
- [Mathematical exploration and discovery at scale](https://mlsanity.com/paper/2511.02864/) (2025)
- [AI Research Agents for Machine Learning: Search, Exploration, and Generalization in MLE-bench](https://mlsanity.com/paper/2507.02554/) (2025)
- [CodexGraph: Bridging Large Language Models and Code Repositories via Code Graph Databases](https://mlsanity.com/paper/2408.03910/) (2024)
- [ML-Master: Towards AI-for-AI via Integration of Exploration and Reasoning](https://mlsanity.com/paper/2506.16499/) (2025)
- [MLE-STAR: Machine Learning Engineering Agent via Search and Targeted Refinement](https://mlsanity.com/paper/2506.15692/) (2025)
- [A Survey on the Optimization of Large Language Model-based Agents](https://mlsanity.com/paper/2503.12434/) (2025)
- [R&D-Agent: An LLM-Agent Framework Towards Autonomous Data Science](https://mlsanity.com/paper/2505.14738/) (2025)
- [Toward Ultra-Long-Horizon Agentic Science: Cognitive Accumulation for Machine Learning Engineering](https://mlsanity.com/paper/2601.10402/) (2026)
- [InternAgent-1.5: A Unified Agentic Framework for Long-Horizon Autonomous Scientific Discovery](https://mlsanity.com/paper/2602.08990/) (2026)
- [MARS: Modular Agent with Reflective Search for Automated AI Research](https://mlsanity.com/paper/2602.02660/) (2026)
- [MLZero: A Multi-Agent System for End-to-end Machine Learning Automation](https://mlsanity.com/paper/2505.13941/) (2025)
- [The FM Agent](https://mlsanity.com/paper/2510.26144/) (2025)
- [Structured Scaling of AI Discovery Across Diverse Scientific Domains](https://mlsanity.com/paper/2604.19341/) (2026)
- [AutoMind: Adaptive Knowledgeable Agent for Automated Data Science](https://mlsanity.com/paper/2506.10974/) (2025)
- [AutoMLGen: Navigating Fine-Grained Optimization for Coding Agents](https://mlsanity.com/paper/2510.08511/) (2025)
- [AIBuildAI: An AI Agent for Automatically Building AI Models](https://mlsanity.com/paper/2604.14455/) (2026)
- [KAPSO: A Knowledge-grounded framework for Autonomous Program Synthesis and Optimization](https://mlsanity.com/paper/2601.21526/) (2026)

## Built on by (most cited first, 28 of 28)

- [Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops](https://mlsanity.com/paper/2607.07663/) (2026)
- [Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agent](https://mlsanity.com/paper/2606.30616/) (2026)
- [Dream-RSI: Recursive Self-Improvement through Evolving Worlds](https://mlsanity.com/paper/2609.14858/) (2026)
- [AgentX: Towards Agent-Driven Self-Iteration of Industrial Recommender Systems](https://mlsanity.com/paper/2606.26859/) (2026)
- [Frontis-MA1: Training an AI4AI Model towards Recursive Self-Improvement in Machine Learning Engineering](https://mlsanity.com/paper/2607.28568/) (2026)
- [ScienceFlow: A long-horizon agent for ML research, scientific discovery and beyond](https://mlsanity.com/paper/2608.14354/) (2026)
- [Agents-K1: Towards Agent-native Knowledge Orchestration](https://mlsanity.com/paper/2606.13669/) (2026)
- [Towards Autonomous and Auditable Medical Imaging Model Development](https://mlsanity.com/paper/2607.10522/) (2026)
- [Recovering Wasted Compute in Autoresearch Agents](https://mlsanity.com/paper/2608.10424/) (2026)
- [What is Missing from AI Post-Training AI: An Empirical Analysis](https://mlsanity.com/paper/2608.19072/) (2026)
- [Repo-To-Skill: Distilling GitHub Repositories Into AI4AI Skills](https://mlsanity.com/paper/2609.02749/) (2026)
- [SciDataSailor: Deep Scientific Data Exploring](https://mlsanity.com/paper/2607.28098/) (2026)
- [When AI Designs AI: Innovation or Imitation?](https://mlsanity.com/paper/2608.17471/) (2026)
- [TraceML: An Empirical Analysis of Human-Agent Planning in Machine Learning Development](https://mlsanity.com/paper/2608.26086/) (2026)
- [AIBuildAI-2.5: Efficient Autonomous AI Model Development Through LLM-Guided Tree Search](https://mlsanity.com/paper/2609.25047/) (2026)
- [EvoTreeNAD: Genealogy-Guided Evolution for LLM-Driven Neural Architecture Discovery](https://mlsanity.com/paper/2609.29016/) (2026)
- [iCoder-27B: Recursive AI-Led Development of Frontier Industrial Coding Model](https://mlsanity.com/paper/2609.29626/) (2026)
- [Advancing Model Research in AgentX: Long-Horizon Autonomy for Industrial Recommender Systems](https://mlsanity.com/paper/2609.30001/) (2026)
- [BioDyad: Synchronize Biomedical Discovery and Machine Learning Engineering](https://mlsanity.com/paper/2609.31939/) (2026)
- [TabFM-Auto: Self-Evolving Pipelines for Tabular Foundation Models](https://mlsanity.com/paper/2609.37989/) (2026)
- [How Much of a Harness Does a Strong Agent Need for Autonomous ML Engineering?](https://mlsanity.com/paper/2609.40303/) (2026)
- [Assembling Insights for Agentic Machine Learning Engineering Systems](https://mlsanity.com/paper/2610.04927/) (2026)
- [JANUS: Enhancing Long-Horizon Autonomous Research with Co-Evolving World Models](https://mlsanity.com/paper/2610.13194/) (2026)
- [Humanize MLE Agents: Multi-Agent Session Alternation Beyond Single-Agent Limits](https://mlsanity.com/paper/2610.13494/) (2026)
- [The Silent Failures in Agentic Data Science](https://mlsanity.com/paper/2610.14115/) (2026)
- [ML Engineering Agents - A Data Management Perspective](https://mlsanity.com/paper/2610.16919/) (2026)
- [Failure Recovery via Belief Updates and World Modeling for Long-Horizon Agents](https://mlsanity.com/paper/2610.18082/) (2026)
- [SwarmRSI: Recursive Self-Improvement through Evolving Scientific Agent Swarm](https://mlsanity.com/paper/2610.20431/) (2026)

## Abstract (arXiv)

Large language model (LLM) agents are increasingly applied to long-horizon tasks such as scientific discovery and machine learning engineering (MLE), where sustained self-evolution becomes a key capability. However, existing MLE agents suffer from inter-branch information isolation, memoryless search, and lack of hierarchical control, which together hinder long-horizon optimization. We present MLEvolve, an LLM-based self-evolving multi-agent framework for end-to-end machine learning algorithm discovery. By extending tree search to Progressive MCGS, MLEvolve enables cross-branch information flow through graph-based reference edges and gradually shifts the search from broad exploration to focused exploitation with an entropy-inspired progressive schedule. To allow the agent to evolve with accumulated experience, we introduce Retrospective Memory, which combines a cold-start domain knowledge base with a dynamic global memory for task-specific experience retrieval and reuse. For stable long-horizon iteration, we further decouple strategic planning from code generation with adaptive coding modes. Evaluation on MLE-Bench shows that MLEvolve achieves state-of-the-art performance across multiple dimensions including average medal rate and valid submission rate under a 12-hour budget (half the standard runtime). Moreover, MLEvolve also outperforms specialized algorithm discovery methods including AlphaEvolve on mathematical algorithm optimization tasks, demonstrating strong cross-domain generalization. Our code is available at https://github.com/InternScience/MLEvolve.