mlsanityData to 14 Oct 2026

ExpeL: LLM Agents Are Experiential Learners

arXiv 2308.10144 ↗First posted 20 Aug 2023cs.LG cs.AI cs.CLcited by 876

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

The recent surge in research interest in applying large language models (LLMs) to decision-making tasks has flourished by leveraging the extensive world knowledge embedded in LLMs. While there is a growing demand to tailor LLMs for custom decision-making tasks, finetuning them for specific tasks is resource-intensive and may diminish the model's generalization capabilities. Moreover, state-of-the-art language models like GPT-4 and Claude are primarily accessible through API calls, with their parametric weights remaining proprietary and unavailable to the public. This scenario emphasizes the growing need for new methodologies that allow learning from agent experiences without requiring parametric updates. To address these problems, we introduce the Experiential Learning (ExpeL) agent. Our agent autonomously gathers experiences and extracts knowledge using natural language from a collection of training tasks. At inference, the agent recalls its extracted insights and past experiences to make informed decisions. Our empirical results highlight the robust learning efficacy of the ExpeL agent, indicating a consistent enhancement in its performance as it accumulates experiences. We further explore the emerging capabilities and transfer learning potential of the ExpeL agent through qualitative observations and additional experiments.

Key facts. Cited by 876 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 50 of its 73 references that are in the map. First posted on arXiv 20 Aug 2023.

Builds on

50 of 73 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Tasks27 May
  2. 2023AdaPlanner: Adaptive Planning from Feedback with Language Models26 May

Other references · 48

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  7. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  9. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  10. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  11. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  12. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
36 more
  1. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  2. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  3. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  4. 2017Billion-scale similarity search with GPUscited by 2,497
  5. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  6. 2019Language Models as Knowledge Bases?cited by 1,936
  7. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  8. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
  9. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,251
  10. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  11. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
  12. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
  13. 2019A Comprehensive Survey on Transfer Learningcited by 1,160
  14. 2015Prioritized Experience Replaycited by 1,099
  15. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,035
  16. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 981
  17. 2023Reasoning with Language Model is Planning with World Modelcited by 861
  18. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 688
  19. 2021Learning To Retrieve Prompts for In-Context Learningcited by 648
  20. 2023ChemCrow: Augmenting large-language models with chemistry toolscited by 631
  21. 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Actioncited by 517
  22. 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 353
  23. 2023EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thoughtcited by 279
  24. 2023Text2Motion: From Natural Language Instructions to Feasible Planscited by 224
  25. 2023TidyBot: Personalized Robot Assistance with Large Language Modelscited by 221
  26. 2023Large Language Models as General Pattern Machinescited by 197
  27. 2023REFLECT: Summarizing Robot Experiences for Failure Explanation and Correctioncited by 175
  28. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 164
  29. 2023Scaling Up and Distilling Down: Language-Guided Robot Skill Acquisitioncited by 159
  30. 2022A Survey on Retrieval-Augmented Text Generationcited by 158
  31. 2023Focused Transformer: Contrastive Training for Context Scalingcited by 133
  32. 2023Retroformer: Retrospective Large Language Agents with Policy Gradient Optimizationcited by 112
  33. 2022Shortcut Learning of Large Language Models in Natural Language Understandingcited by 85
  34. 2023From Pixels to UI Actions: Learning to Follow Instructions via Graphical User Interfacescited by 68
  35. 2023Learning to Retrieve In-Context Examples for Large Language Modelscited by 60
  36. 2022Large-Scale Retrieval for Reinforcement Learningcited by 24

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 15; 2024: 103; 2025: 126; 2026: 632.

876 papers.

Sort:

By area:

  1. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  2. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  3. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 832
  4. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 634
  5. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 344
  6. 2025ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memorycited by 284
  7. 2026SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learningcited by 278
  8. 2024TextGrad: Automatic "Differentiation" via Textcited by 273
  9. 2024Scaling Large Language Model-based Multi-Agent Collaborationcited by 241
  10. 2024AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environmentscited by 231
  11. 2024Data Interpreter: An LLM Agent For Data Sciencecited by 225
  12. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 205
38 more
  1. 2024Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaborationcited by 189
  2. 2025Memory Injection Attacks on LLM Agents via Query-Only Interactioncited by 144
  3. 2026MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agentscited by 142
  4. 2026Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnessescited by 142
  5. 2023GPT-4V in Wonderland: Large Multimodal Models for Zero-Shot Smartphone GUI Navigationcited by 137
  6. 2024Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospectscited by 132
  7. 2024OASIS: Open Agent Social Interaction Simulations with One Million Agentscited by 132
  8. 2025Memento: Fine-tuning LLM Agents without Fine-tuning LLMscited by 127
  9. 2025A Survey of Context Engineering for Large Language Modelscited by 126
  10. 2024DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoningcited by 125
  11. 2025G-Memory: Tracing Hierarchical Memory for Multi-Agent Systemscited by 122
  12. 2024FinCon: A Synthesized LLM Multi-Agent System with Conceptual Verbal Reinforcement for Enhanced Financial Decision Makingcited by 113
  13. 2024A Survey on Large Language Model-Based Game Agentscited by 112
  14. 2024HiAgent: Hierarchical Working Memory Management for Solving Long-Horizon Agent Tasks with Large Language Modelcited by 106
  15. 2023Avalon's Game of Thoughts: Battle Against Deception through Recursive Contemplationcited by 104
  16. 2025How Memory Management Impacts LLM Agents: An Empirical Study of Experience-Following Behaviorcited by 101
  17. 2024A Review of Large Language Models and Autonomous Agents in Chemistrycited by 99
  18. 2025MemEvolve: Meta-Evolution of Agent Memory Systemscited by 95
  19. 2024Agent-Pro: Learning to Evolve via Policy-Level Reflection and Optimizationcited by 94
  20. 2026SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalizationcited by 94
  21. 2026Skill-SD: Skill-Conditioned Self-Distillation for Multi-turn LLM Agentscited by 90
  22. 2024From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Futurecited by 86
  23. 2025MemInsight: Autonomous Memory Augmentation for LLM Agentscited by 85
  24. 2024AgentGen: Enhancing Planning Abilities for Large Language Model based Agent via Environment and Task Generationcited by 83
  25. 2025Lifelong Learning of Large Language Model based Agents: A Roadmapcited by 83
  26. 2025Agent-R: Training Language Model Agents to Reflect via Iterative Self-Trainingcited by 82
  27. 2026SkillClaw: Let Skills Evolve Collectively with Agentic Evolvercited by 81
  28. 2024RAP: Retrieval-Augmented Planning with Contextual Memory for Multimodal LLM Agentscited by 80
  29. 2023Put Your Money Where Your Mouth Is: Evaluating Strategic Planning and Execution of LLM Agents in an Auction Arenacited by 78
  30. 2024EHRAgent: Code Empowers Large Language Models for Few-shot Complex Tabular Reasoning on Electronic Health Recordscited by 78
  31. 2024Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systemscited by 76
  32. 2026SkillX: Automatically Constructing Skill Knowledge Bases for Agentscited by 76
  33. 2026XSkill: Continual Learning from Experience and Skills in Multimodal Agentscited by 75
  34. 2024AutoGuide: Automated Generation and Selection of Context-Aware Guidelines for Large Language Model Agentscited by 74
  35. 2025Remember Me, Refine Me: A Dynamic Procedural Memory Framework for Experience-Driven Agent Evolutioncited by 73
  36. 2024AvaTaR: Optimizing LLM Agents for Tool Usage via Contrastive Reasoningcited by 72
  37. 2024Agent Planning with World Knowledge Modelcited by 70
  38. 2025SWE-Exp: Experience-Driven Software Issue Resolutioncited by 65

50 most cited of 876 All 876 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026ExpGraph: Model-Agnostic Experience Learning with Graph-Structured Memory for LLM Agents
  2. 2026ExpWeaver: LLM Agents Learn from Experience via Latent RAG
  3. 2024Can large language models explore in-context?
  4. 2026Pushing the Limits of LLM Tool Calling via Experiential Knowledge Integration and Activation
  5. 2025From Language to Action: A Review of Large Language Models as Autonomous Agents and Tool Users
  6. 2024Controlling Large Language Model Agents with Entropic Activation Steering
  7. 2025Large Language Models Think Too Fast To Explore Effectively
  8. 2025Expanding the Action Space of LLMs to Reason Beyond Language