# Dream-RSI: Recursive Self-Improvement through Evolving Worlds

arXiv: https://arxiv.org/abs/2609.14858 · HTML: https://mlsanity.com/paper/2609.14858/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 61 references that are in the map.
- First posted on arXiv 14 Sep 2026.

## Builds on (most cited first, 30 of 44)

- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [Mastering Diverse Domains through World Models](https://mlsanity.com/paper/2301.04104/) (2023)
- [Mastering Atari with Discrete World Models](https://mlsanity.com/paper/2010.02193/) (2020)
- [AlphaEvolve: A coding agent for scientific and algorithmic discovery](https://mlsanity.com/paper/2506.13131/) (2025)
- [Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces](https://mlsanity.com/paper/2601.11868/) (2026)
- [Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models](https://mlsanity.com/paper/2510.04618/) (2025)
- [Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents](https://mlsanity.com/paper/2505.22954/) (2025)
- [Meta-Harness: End-to-End Optimization of Model Harnesses](https://mlsanity.com/paper/2603.28052/) (2026)
- [Memory in the Age of AI Agents](https://mlsanity.com/paper/2512.13564/) (2025)
- [ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory](https://mlsanity.com/paper/2509.25140/) (2025)
- [R-Zero: Self-Evolving Reasoning LLM from Zero Data](https://mlsanity.com/paper/2508.05004/) (2025)
- [KernelBench: Can LLMs Write Efficient GPU Kernels?](https://mlsanity.com/paper/2502.10517/) (2025)
- [Cognitive Mapping and Planning for Visual Navigation](https://mlsanity.com/paper/1702.03920/) (2017)
- [ShinkaEvolve: Towards Open-Ended And Sample-Efficient Program Evolution](https://mlsanity.com/paper/2509.19349/) (2025)
- [Training Agents Inside of Scalable World Models](https://mlsanity.com/paper/2509.24527/) (2025)
- [CoEvoSkills: Self-Evolving Agent Skills via Co-Evolutionary Verification](https://mlsanity.com/paper/2604.01687/) (2026)
- [Learning to Discover at Test Time](https://mlsanity.com/paper/2601.16175/) (2026)
- [Mathematical exploration and discovery at scale](https://mlsanity.com/paper/2511.02864/) (2025)
- [ThetaEvolve: Test-time Learning on Open Problems](https://mlsanity.com/paper/2511.23473/) (2025)
- [Parallel-R1: Towards Parallel Thinking via Reinforcement Learning](https://mlsanity.com/paper/2509.07980/) (2025)
- [EvoX: Meta-Evolution for Automated Discovery](https://mlsanity.com/paper/2602.23413/) (2026)
- [SkillOS: Learning Skill Curation for Self-Evolving Agents](https://mlsanity.com/paper/2605.06614/) (2026)
- [CodeEvolve: an open source evolutionary coding agent for algorithmic discovery and optimization](https://mlsanity.com/paper/2510.14150/) (2025)
- [Long-Horizon-Terminal-Bench: Testing the Limits of Agents on Long-Horizon Terminal Tasks with Dense Reward-Based Grading](https://mlsanity.com/paper/2607.08964/) (2026)
- [MLEvolve: A Self-Evolving Framework for Automated Machine Learning Algorithm Discovery](https://mlsanity.com/paper/2606.06473/) (2026)
- [DeltaEvolve: Accelerating Scientific Discovery through Momentum-Driven Evolution](https://mlsanity.com/paper/2602.02919/) (2026)
- [PACEvolve: Enabling Progress-Aware Consistent Evolution](https://mlsanity.com/paper/2601.10657/) (2026)
- [K-Search: LLM Kernel Generation via Co-Evolving Intrinsic World Model](https://mlsanity.com/paper/2602.19128/) (2026)
- [MetaSkill-Evolve: Recursive Self-Improvement of LLM Agents via Two-Timescale Meta-Skill Evolution](https://mlsanity.com/paper/2607.05297/) (2026)

## Built on by (most cited first, 18 of 18)

- [Control the Harness, Control the Cost: Routing and Governing AI Coding Agents in the Enterprise](https://mlsanity.com/paper/2609.28919/) (2026)
- [Scaling Trajectories for Complex Tasks through Recursive Self-Rewrite](https://mlsanity.com/paper/2610.02826/) (2026)
- [Advancing Model Research in AgentX: Long-Horizon Autonomy for Industrial Recommender Systems](https://mlsanity.com/paper/2609.30001/) (2026)
- [ReVision3D: Attribution-Guided Recursive Self-Improvement for 3D Medical Perception](https://mlsanity.com/paper/2609.32984/) (2026)
- [Can Terminal Agents Trust Their Own Verification? Diagnosing and Improving Self-Verification](https://mlsanity.com/paper/2609.38812/) (2026)
- [Experimental Experience Modeling for Autonomous Research](https://mlsanity.com/paper/2609.39392/) (2026)
- [Towards Better Exploration in Sequential Test-Time Scaling](https://mlsanity.com/paper/2609.39632/) (2026)
- [LabBook: Harnessing Experimental History for Efficient LLM-Driven Discovery](https://mlsanity.com/paper/2610.00675/) (2026)
- [Align Then Reason: A Multimodal Lip-Sync Judge for Dubbing](https://mlsanity.com/paper/2610.00825/) (2026)
- [MMPostTrainBench: Benchmarking Autonomous Research for Multimodal Post-Training](https://mlsanity.com/paper/2610.05398/) (2026)
- [CARE: Certifying Acceleration for Vision-Language-Action Inference](https://mlsanity.com/paper/2610.08917/) (2026)
- [PhysEvo: Astra Can Act, Let It](https://mlsanity.com/paper/2610.08995/) (2026)
- [From Pareto to Preference: Personalized Test-Time Scaling via Amortized Agentic Policy Discovery](https://mlsanity.com/paper/2610.09684/) (2026)
- [Trace-to-Architecture: Capability Bottleneck Diagnosis for Harness Revision](https://mlsanity.com/paper/2610.16725/) (2026)
- [MedEvo: Medical Reasoning Evolution through Knowledge Navigation](https://mlsanity.com/paper/2610.17690/) (2026)
- [ECG-RSI: Recursive Self-Improvement through Targeted Electrocardiogram Waveform Evolution](https://mlsanity.com/paper/2610.17718/) (2026)
- [AReaL-RSI: Adaptive Scaling and Coordination of Recursive Self-Improvement](https://mlsanity.com/paper/2610.18083/) (2026)
- [SwarmRSI: Recursive Self-Improvement through Evolving Scientific Agent Swarm](https://mlsanity.com/paper/2610.20431/) (2026)

## Abstract (arXiv)

Recursive self-improvement is becoming essential for autonomous AI agents, whose progress depends on discovering high-value solutions across complex domains. Effective exploration drives this process, yet managing and improving exploration strategies remains a major bottleneck. Current systems face a fundamental dilemma: fixed strategies fail to adapt as search spaces scale, while online policy optimization must navigate vast meta-search spaces under delayed, expensive feedback from long-horizon rollouts. We introduce textscDream-RSI, a framework for scalable, recursively self-improving exploration. A lightweight orchestration layer makes exploration explicit and programmable while leaving the underlying base agent unchanged. Our key insight is that accumulated discovery history can act as a replay simulator over the realized search space. By dreaming within this simulator built from historical discovery trees, textscDream-RSI obtains immediate, low-cost off-policy feedback to evaluate and refine exploration policies without repeated, expensive online evaluation. The improved policy is then redeployed online to drive further discovery, continuously expanding the simulator pool in a self-improving loop. Across 9 tasks in 4 domains, textscDream-RSI achieves competitive quality and improves discovery efficiency in several settings.