mlsanityData to 14 Oct 2026

Agent2: An Agent-Generates-Agent Framework for Reinforcement Learning Automation

arXiv 2509.13368 ↗First posted 16 Sep 2025cs.AI cs.LGcited by 0

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

also in 1 more
Abstract (arXiv)

Reinforcement learning (RL) agent development traditionally requires substantial expertise and iterative effort, often leading to high failure rates and limited accessibility. This paper introduces Agent2, an LLM-driven agent-generates-agent framework for fully automated RL agent design. Agent2 autonomously translates natural language task descriptions and environment code into executable RL solutions without human intervention. The framework adopts a dual-agent architecture: a Generator Agent that analyzes tasks and designs agents, and a Target Agent that is automatically generated and executed. To better support automation, RL development is decomposed into two stages, MDP modeling and algorithmic optimization, facilitating targeted and effective agent generation. Built on the Model Context Protocol, Agent2 provides a unified framework for standardized agent creation across diverse environments and algorithms, incorporating adaptive training management and intelligent feedback analysis for continuous refinement. Extensive experiments on benchmarks including MuJoCo, MetaDrive, MPE, and SMAC show that Agent2 outperforms manually designed baselines across all tasks, achieving up to 55% performance improvement with consistent average gains. By enabling a closed-loop, end-to-end automation pipeline, this work advances a new paradigm in which agents can design and optimize other agents, underscoring the potential of agent-generates-agent systems for automated AI development.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 27 of its 41 references that are in the map. First posted on arXiv 16 Sep 2025.

Builds on

27 of 41 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025LERO: LLM-driven Evolutionary framework with Hybrid Rewards and Enhanced Observation for Multi-Agent Reinforcement Learning25 Mar
  2. 2025Leveraging Large Language Models for Effective and Explainable Multi-Agent Credit Assignment24 Feb
  3. 2025Speaking the Language of Teamwork: LLM-Guided Credit Assignment in Multi-Agent Reinforcement Learning6 Feb
  4. 2024YOLO-MARL: You Only LLM Once for Multi-Agent Reinforcement Learning5 Oct
  5. 2024LLM-based Multi-Agent Reinforcement Learning: Current and Future Directions17 May

Other references · 22

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  3. 2019The StarCraft Multi-Agent Challengecited by 597
  4. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 571
  5. 2019AutoML: A Survey of the State-of-the-Artcited by 390
  6. 2017Deep Reinforcement Learning: An Overviewcited by 333
  7. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learningcited by 290
  8. 2021Tianshou: a Highly Modularized Deep Reinforcement Learning Librarycited by 143
  9. 2023EvoPrompting: Language Models for Code-Level Neural Architecture Searchcited by 131
  10. 2024Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methodscited by 116
  11. 2023LLMatic: Neural Architecture Search via Large Language Models and Quality Diversity Optimizationcited by 87
  12. 2023Using Large Language Models for Hyperparameter Optimizationcited by 71
10 more
  1. 2024Large Language Model Agent for Hyper-Parameter Optimizationcited by 64
  2. 2023Auto MC-Reward: Automated Dense Reward Design with Large Language Models for Minecraftcited by 61
  3. 2023Self-Refined Large Language Model as Automated Reward Function Designer for Deep Reinforcement Learning in Roboticscited by 36
  4. 2024LLM-Empowered State Representation for Reinforcement Learningcited by 28
  5. 2024Sequential Large Language Model-Based Hyper-parameter Optimizationcited by 24
  6. 2024ExploRLLM: Guiding Exploration in Reinforcement Learning with Large Language Modelscited by 19
  7. 2023XuanCe: A Comprehensive and Unified Deep Reinforcement Learning Librarycited by 5
  8. 2022ARLO: A Framework for Automated Reinforcement Learningcited by 3
  9. 2025Combining LLM decision and RL action selection to improve RL policy for adaptive interventionscited by 2
  10. 2025CAMEL: Continuous Action Masking Enabled by Large Language Models for Reinforcement Learningcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Agent-R1: A Unified and Modular Framework for Agentic Reinforcement Learning
  2. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Survey
  3. 2025AutoForge: Automated Environment Synthesis for Agentic Reinforcement Learning
  4. 2025AgentFly: Extensible and Scalable Reinforcement Learning for LM Agents
  5. 2026AutoGym: Blueprint-First Generation of Verifiable Agent Gyms
  6. 2024Game On: Towards Language Models as RL Experimenters
  7. 2025A-LAMP: Agentic LLM-Based Framework for Automated MDP Modeling and Policy Generation
  8. 2026Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligence