# Recursive Agent Optimization

arXiv: https://arxiv.org/abs/2605.06639 · HTML: https://mlsanity.com/paper/2605.06639/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods.
- Builds on 20 of its 29 references that are in the map.
- First posted on arXiv 7 May 2026.

## Placed in

- Deep reinforcement learning methods / [RL training of language and agent models](https://mlsanity.com/deeprl/rl-training-of-language-and-agent-models/)

## Builds on (most cited first, 20 of 20)

- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [OpenAI GPT-5 System Card](https://mlsanity.com/paper/2601.03267/) (2025)
- [Executable Code Actions Elicit Better LLM Agents](https://mlsanity.com/paper/2402.01030/) (2024)
- [Kimi K2.5: Visual Agentic Intelligence](https://mlsanity.com/paper/2602.02276/) (2026)
- [Data-Efficient Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1805.08296/) (2018)
- [MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention](https://mlsanity.com/paper/2506.13585/) (2025)
- [AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning](https://mlsanity.com/paper/2505.24298/) (2025)
- [Chain of Agents: Large Language Models Collaborating on Long-Context Tasks](https://mlsanity.com/paper/2406.02818/) (2024)
- [ADaPT: As-Needed Decomposition and Planning with Language Models](https://mlsanity.com/paper/2311.05772/) (2023)
- [Scaling Long-Horizon LLM Agent via Context-Folding](https://mlsanity.com/paper/2510.11967/) (2025)
- [Learning with AMIGo: Adversarially Motivated Intrinsic Goals](https://mlsanity.com/paper/2006.12122/) (2020)
- [DeepDive: Advancing Deep Search Agents with Knowledge Graphs and Multi-Turn RL](https://mlsanity.com/paper/2509.10446/) (2025)
- [Oolong: Evaluating Long Context Reasoning and Aggregation Capabilities](https://mlsanity.com/paper/2511.02817/) (2025)
- [THREAD: Thinking Deeper with Recursive Spawning](https://mlsanity.com/paper/2405.17402/) (2024)
- [Effective Strategies for Asynchronous Software Engineering Agents](https://mlsanity.com/paper/2603.21489/) (2026)
- [Reasoning Cache: Continual Improvement Over Long Horizons via Short-Horizon RL](https://mlsanity.com/paper/2602.03773/) (2026)
- [ReDel: A Toolkit for LLM-Powered Recursive Multi-Agent Systems](https://mlsanity.com/paper/2408.02248/) (2024)
- [QED-Nano: Teaching a Tiny Model to Prove Hard Theorems](https://mlsanity.com/paper/2604.04898/) (2026)
- [Think, But Don't Overthink: Reproducing Recursive Language Models](https://mlsanity.com/paper/2603.02615/) (2026)

## Built on by (most cited first, 5 of 5)

- [Multi-Agent Computer Use](https://mlsanity.com/paper/2606.01533/) (2026)
- [Mint-Agent: Introducing Finance-Native Agentic Foundation Models](https://mlsanity.com/paper/2608.16386/) (2026)
- [Recursive Agentic Reasoning](https://mlsanity.com/paper/2608.23956/) (2026)
- [Hermes: Learning Contextual Reasoning Unlocks Test-Time Scaling](https://mlsanity.com/paper/2609.38332/) (2026)
- [Self-Evaluating Recursive Agents](https://mlsanity.com/paper/2610.04902/) (2026)

## Abstract (arXiv)

We introduce Recursive Agent Optimization (RAO), a reinforcement learning approach for training recursive agents: agents that can spawn and delegate sub-tasks to new instantiations of themselves recursively. Recursive agents implement an inference-time scaling algorithm that naturally allows agents to scale to longer contexts and generalize to more difficult problems via divide-and-conquer. RAO provides a method to train models to best take advantage of such recursive inference, teaching agents when and how to delegate and communicate. We find that recursive agents trained in this way enjoy better training efficiency, can scale to tasks that go beyond the model's context window, generalize to tasks much harder than the ones the agent was trained on, and can enjoy reduced wall-clock time compared to single-agent systems.