mlsanityData to 7 Oct 2026

AREX-2: Advancing Self-Improving Agents through Long-Horizon Reflective Tasks

arXiv 2609.38288 ↗First posted 29 Sep 2026cs.AIcited by 0

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks also fits: Reinforcement learning for LLM agents

also in 1 more
Abstract (arXiv)

We present AREX-2, an effort to advance the self-improving capability of LLM agents, which we define as the ability to iteratively refine a solution at test time. This ability rests on two complementary capabilities: reflection, which produces a solution better than the current one, and long-horizon execution, which keeps the iteration effective over many rounds. We hypothesize that both capabilities are domain-agnostic, and can therefore be learned in scenarios that are well suited for supervision. Accordingly, we synthesize long-horizon improvement trajectories from machine learning and algorithmic programming tasks, two domains that offer verifiable feedback and reward sustained iteration. Trained on this data, our agent, built on Qwen3.8-27B, achieves strong results on MLE-bench Lite (81.8) and Frontier-CS (70.7), transfers to deep research with 84.0 on BrowseComp, 52.6 on HLE, 92.2 on GAIA, and 93.8 on DeepSearchQA, and keeps improving as its budget of rounds grows. These results show that long-horizon reflective data is an effective route toward self-improving agents.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 16 of its 18 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

16 of 18 references.

References · 16

  1. 2023GAIA: a benchmark for General AI Assistantscited by 1,084
  2. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,002
  3. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
  4. 2025AlphaEvolve: A coding agent for scientific and algorithmic discoverycited by 763
  5. 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 616
  6. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 486
  7. 2024MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineeringcited by 429
  8. 2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Modelscited by 417
  9. 2024Training Software Engineering Agents and Verifiers with SWE-Gymcited by 300
  10. 2025SWE-smith: Scaling Data for Software Engineering Agentscited by 255
  11. 2025AIDE: AI-Driven Exploration in the Space of Codecited by 206
  12. 2025R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agentscited by 140
4 more
  1. 2026DeepSearchQA: Bridging the Comprehensiveness Gap for Deep Research Agentscited by 44
  2. 2025FrontierCS: Evolving Challenges for Evolving Intelligencecited by 19
  3. 2026Frontis-MA1: Training an AI4AI Model towards Recursive Self-Improvement in Machine Learning Engineeringcited by 7
  4. 2026AREX: Towards a Recursively Self-Improving Agent for Deep Researchcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026MetaSkill-Evolve: Recursive Self-Improvement of LLM Agents via Two-Timescale Meta-Skill Evolution
  2. 2026RRSI: Regularized Recursive Self-Improvement of Agent Harnesses
  3. 2026Metan: Recursive Self-Improvement through Emergent Depth
  4. 2026Recursive self-improvement of AI research agents
  5. 2026AI4AI-Bench: Benchmarking LLM Agents in Algorithmic Design for Recursive Self-Improvement
  6. 2026RSIBench-Data: Benchmarking Data-Centric Research for Recursive Self-Improvement
  7. 2026Experiential Reflective Learning for Self-Improving LLM Agents
  8. 2025Reflect, Retry, Reward: Self-Improving LLMs via Reinforcement Learning