mlsanityData to 7 Oct 2026

AGILE: A Novel Reinforcement Learning Framework of LLM Agents

arXiv 2405.14751 ↗First posted 23 May 2024cs.LGcited by 45

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

We introduce a novel reinforcement learning framework of LLM agents named AGILE (AGent that Interacts and Learns from Environments) designed to perform complex conversational tasks with users, leveraging LLMs, memory, tools, and interactions with experts. The agent possesses capabilities beyond conversation, including reflection, tool usage, and expert consultation. We formulate the construction of such an LLM agent as a reinforcement learning (RL) problem, in which the LLM serves as the policy model. We fine-tune the LLM using labeled data of actions and the PPO algorithm. We focus on question answering and release a dataset for agents called ProductQA, comprising challenging questions in online shopping. Our extensive experiments on ProductQA, MedMCQA and HotPotQA show that AGILE agents based on 7B and 13B LLMs trained with PPO can outperform GPT-4 agents. Our ablation study highlights the indispensability of memory, tools, consultation, reflection, and reinforcement learning in achieving the agent's strong performance. Datasets and code are available at https://github.com/bytarnish/AGILE.

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 43 of its 56 references that are in the map. First posted on arXiv 23 May 2024.

Builds on

43 of 56 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Voyager: An Open-Ended Embodied Agent with Large Language Models25 May
  2. 2023Reflexion: Language Agents with Verbal Reinforcement Learning20 Mar

Other references · 41

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  7. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  8. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  9. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  10. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  11. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  12. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
29 more
  1. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  2. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  3. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  4. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  5. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,210
  6. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  7. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  8. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  9. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
  10. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  11. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 814
  12. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
  13. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
  14. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  15. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 630
  16. 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 525
  17. 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Actioncited by 511
  18. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 452
  19. 2023Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agentscited by 396
  20. 2022ScienceWorld: Is your Agent Smarter than a 5th Grader?cited by 298
  21. 2023Robots That Ask For Help: Uncertainty Alignment for Large Language Model Plannerscited by 298
  22. 2023AutoAgents: A Framework for Automatic Agent Generationcited by 211
  23. 2024RAP: Retrieval-Augmented Planning with Contextual Memory for Multimodal LLM Agentscited by 80
  24. 2024Tell Me More! Towards Implicit User Intention Understanding of Language Model Driven Agentscited by 71
  25. 2023Ask an Expert: Leveraging Language Models to Improve Strategic Reasoning in Goal-Oriented Dialogue Modelscited by 30
  26. 2024Small Language Models Learn Enhanced Reasoning Skills from Medical Textbookscited by 30
  27. 2023LLM A*: Human in the Loop Large Language Models Enabled A* Search for Roboticscited by 26
  28. 2023Asking Before Acting: Gather Information in Embodied Decision Making with Language Modelscited by 12
  29. 2023Product Question Answering in E-Commerce: A Surveycited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 24; 2026: 21.

45 papers.

Sort:

By area:

  1. 2025Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memorycited by 88
  2. 2025PaSa: An LLM Agent for Comprehensive Academic Paper Searchcited by 61
  3. 2025A Survey on the Optimization of Large Language Model-based Agentscited by 45
  4. 2025OS Agents: A Survey on MLLM-based Agents for General Computing Devices Usecited by 35
  5. 2025Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecyclecited by 34
  6. 2025Orak: A Foundational Benchmark for Training and Evaluating LLM Agents on Diverse Video Gamescited by 30
  7. 2025Graphs Meet AI Agents: Taxonomy, Progress, and Future Opportunitiescited by 23
  8. 2025Heterogeneous Swarms: Jointly Optimizing Model Roles and Weights for Multi-LLM Systemscited by 22
  9. 2025PALADIN: Self-Correcting Language Model Agents to Cure Tool-Failure Casescited by 21
  10. 2025Iterative Tool Usage Exploration for Multimodal Agents via Step-wise Preference Tuningcited by 18
  11. 2025Learning to Deliberate: Meta-policy Collaboration for Agentic LLMs with Multi-agent Reinforcement Learningcited by 18
  12. 2026Auditing Multi-Agent LLM Reasoning Trees Outperforms Majority Vote and LLM-as-Judgecited by 16
33 more
  1. 2025Towards Transparent and Incentive-Compatible Collaboration in Decentralized LLM Multi-Agent Systems: A Blockchain-Driven Approachcited by 11
  2. 2026ARLArena: A Unified Framework for Stable Agentic Reinforcement Learningcited by 11
  3. 2025The Evolving Landscape of LLM- and VLM-Integrated Reinforcement Learningcited by 10
  4. 2026Adaptive Collaboration with Humans: Metacognitive Policy Optimization for Multi-Agent LLMs with Continual Learningcited by 9
  5. 2025AgentDNS: A Root Domain Naming System for LLM Agentscited by 8
  6. 2026Learning How to Remember: A Meta-Cognitive Management Method for Structured and Transferable Agent Memorycited by 8
  7. 2025VisEscape: A Benchmark for Evaluating Exploration-driven Decision-making in Virtual Escape Roomscited by 6
  8. 2025OS-MAP: How Far Can Computer-Using Agents Go in Breadth and Depth?cited by 6
  9. 2026SEAD: Self-Evolving Agent for Multi-Turn Service Dialoguecited by 6
  10. 2025Language Models can Self-Improve at State-Value Estimation for Better Searchcited by 4
  11. 2025Building LLM Agents by Incorporating Insights from Computer Systemscited by 4
  12. 2025Collision- and Reachability-Aware Multi-Robot Control with Grounded LLM Plannerscited by 4
  13. 2025Multi-level Value Alignment in Agentic AI Systems: Survey and Perspectivescited by 4
  14. 2025Cognitive Agents Powered by Large Language Models for Agile Software Project Managementcited by 4
  15. 2026Can RL Improve Generalization of LLM Agents? An Empirical Studycited by 4
  16. 2026Supervised Fine-Tuning versus Reinforcement Learning: A Study of Post-Training Methods for Large Language Modelscited by 3
  17. 2026MagicAgent: Towards Generalized Agent Planningcited by 2
  18. 2026SAGE: A Service Agent Graph-guided Evaluation Benchmarkcited by 2
  19. 2025Enabling Autonomic Microservice Management through Self-Learning Agentscited by 1
  20. 2025CSP: A Simulator For Multi-Agent Ranking Competitionscited by 1
  21. 2025LLM-Driven Self-Refinement for Embodied Drone Task Planningcited by 1
  22. 2026TodoEvolve: Learning to Architect Agent Planning Systemscited by 1
  23. 2026Reinforcing Real-world Service Agents: Balancing Utility and Cost in Task-oriented Dialoguecited by 1
  24. 2026Hierarchical Reinforcement Learning with Augmented Step-Level Transitions for LLM Agentscited by 1
  25. 2026Beyond Negative Rollouts: Positive-Only Policy Optimization with Implicit Negative Gradientscited by 1
  26. 2026VideoSearch-R1: Iterative Video Retrieval and Reasoning via Soft Query Refinementcited by 1
  27. 2026Decoupling Strategy and Execution in Task-Focused Dialogue via Goal-Oriented Preference Optimizationcited by 0
  28. 2026Agentic Federated Learning: The Future of Distributed Training Orchestrationcited by 0
  29. 2026From Outcomes to Actions: Leveraging Hindsight for Long-Horizon Language Agent Trainingcited by 0
  30. 2026A Diagnostic Framework for AI Agent Behaviorcited by 0
  31. 2026Agent-G2: Gaussian Guidance for Agentic Reinforcement Learningcited by 0
  32. 2026EDGEGEN: Improving Tool-Calling Agents Beyond Happy Paths with Synthetic Edge Case Generationcited by 0
  33. 2026Learning Perturbation Robust Policies for LLM Agents with Stable Optimizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025EvolveR: Self-Evolving LLM Agents through an Experience-Driven Lifecycle
  2. 2026From Question Answering to Task Completion: A Survey on Agent System and Harness Design
  3. 2023ReST meets ReAct: Self-Improvement for Multi-Step Reasoning LLM Agent
  4. 2025Agent-as-Tool: A Study on the Hierarchical Decision Making with Reinforcement Learning
  5. 2026SAGE: Tool-Augmented LLM Task Solving Strategies in Scalable Multi-Agent Environments
  6. 2025Training Proactive and Personalized LLM Agents
  7. 2023Enabling Intelligent Interactions between an Agent and an LLM: A Reinforcement Learning Approach
  8. 2025An Empirical Study on Reinforcement Learning for Reasoning-Search Interleaved LLM Agents