# Scaling Discovery Requires Forgetting

arXiv: https://arxiv.org/abs/2610.20563 · HTML: https://mlsanity.com/paper/2610.20563/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Agent memory systems, one of 16 directions in LLM agents and tool use.
- Builds on 21 of its 30 references that are in the map.
- First posted on arXiv 14 Oct 2026.

## Placed in

- LLM agents and tool use / [Agent memory systems](https://mlsanity.com/agents/agent-memory-systems/)

## Builds on (most cited first, 21 of 21)

- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [ChatDev: Communicative Agents for Software Development](https://mlsanity.com/paper/2307.07924/) (2023)
- [Large Language Models as Optimizers](https://mlsanity.com/paper/2309.03409/) (2023)
- [Why Do Multi-Agent LLM Systems Fail?](https://mlsanity.com/paper/2503.13657/) (2025)
- [GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning](https://mlsanity.com/paper/2507.19457/) (2025)
- [Meta-Harness: End-to-End Optimization of Model Harnesses](https://mlsanity.com/paper/2603.28052/) (2026)
- [Scaling Large Language Model-based Multi-Agent Collaboration](https://mlsanity.com/paper/2406.07155/) (2024)
- [DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression](https://mlsanity.com/paper/2609.19969/) (2026)
- [Talk Isn't Always Cheap: Understanding Failure Modes in Multi-Agent Debate](https://mlsanity.com/paper/2509.05396/) (2025)
- [AdaEvolve: Adaptive LLM Driven Zeroth-Order Optimization](https://mlsanity.com/paper/2602.20133/) (2026)
- [Understanding Agent Scaling in LLM-Based Multi-Agent Systems via Diversity](https://mlsanity.com/paper/2602.03794/) (2026)
- [Multi-Agent Teams Hold Experts Back](https://mlsanity.com/paper/2602.01011/) (2026)
- [ScientistOne: Towards Human-Level Autonomous Research via Chain-of-Evidence](https://mlsanity.com/paper/2605.26340/) (2026)
- [Decentralized Multi-Agent Systems with Shared Context](https://mlsanity.com/paper/2606.10662/) (2026)
- [Scaling Discovery through Test-Time Communication](https://mlsanity.com/paper/2609.21032/) (2026)
- [SwarmResearch: Orchestrating Coding Agents for Open-Ended Discovery](https://mlsanity.com/paper/2607.02807/) (2026)
- [The Interaction Tax: When Communication Erases Diversity in Multi-Agent Teams](https://mlsanity.com/paper/2608.23541/) (2026)
- [When Agents Slow Down: Understanding LLM Agents' Test-Time Strategies via Elo-per-token Analysis](https://mlsanity.com/paper/2609.15309/) (2026)
- [Beneath the Diff: Diagnosing and Mitigating Algorithmic Mode Collapse in Code-Level Autonomous Research Loops](https://mlsanity.com/paper/2609.00077/) (2026)
- [Fork-and-Flush: Escaping Idea Basins in Autoresearch Agents](https://mlsanity.com/paper/2610.07447/) (2026)
- [Stateless Language Agents: Scaling Long-Horizon Automated Research](https://mlsanity.com/paper/2610.07625/) (2026)

## Abstract (arXiv)

Automated research with LLM agents has shown promising results in various domains. Through iterative refinement and scored verifiers, LLM agents are capable of discovering new ideas. There are different ways to scale automated research: let one agent iterate for longer, run several agents in parallel, or further let the parallel agents communicate and build on one another's candidates. However, we find that all three strategies eventually plateau as the agents accumulate history. Each agent becomes anchored to its own past attempts, making fewer and smaller edits. Beyond this point, additional budget brings little further improvement. In this paper, we show that the key to scaling discovery further is to forget. Periodically discarding the search state, including every agent's session and the candidate history, and restarting from the best candidate found so far, unblocks the search and pushes the same budget further. We implement forgetting in a team of agents with communication and evaluate it on four tasks spanning heuristic programming, kernel optimization and model training. The results show up to 15% improvement compared to parallel agents, and up to 39% compared to a single agent. Our results suggest that, for open-ended discovery, what an agent is allowed to remember matters as much as how much compute it is given.