mlsanityData to 7 Oct 2026

Reflexion: Language Agents with Verbal Reinforcement Learning

arXiv 2303.11366 ↗First posted 20 Mar 2023cs.AI cs.CL cs.LGcited by 4,064

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Large language models (LLMs) have been increasingly used to interact with external environments (e.g., games, compilers, APIs) as goal-driven agents. However, it remains challenging for these language agents to quickly and efficiently learn from trial-and-error as traditional reinforcement learning methods require extensive training samples and expensive model fine-tuning. We propose Reflexion, a novel framework to reinforce language agents not by updating weights, but instead through linguistic feedback. Concretely, Reflexion agents verbally reflect on task feedback signals, then maintain their own reflective text in an episodic memory buffer to induce better decision-making in subsequent trials. Reflexion is flexible enough to incorporate various types (scalar values or free-form language) and sources (external or internally simulated) of feedback signals, and obtains significant improvements over a baseline agent across diverse tasks (sequential decision-making, coding, language reasoning). For example, Reflexion achieves a 91% pass@1 accuracy on the HumanEval coding benchmark, surpassing the previous state-of-the-art GPT-4 that achieves 80%. We also conduct ablation and analysis studies using different feedback signals, feedback incorporation methods, and agent types, and provide insights into how they affect performance.

Key facts. Cited by 4,064 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 18 of its 36 references that are in the map. First posted on arXiv 20 Mar 2023.

Builds on

18 of 36 references.

References · 18

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  7. 2021Program Synthesis with Large Language Modelscited by 3,509
  8. 2022Emergent Abilities of Large Language Modelscited by 2,483
  9. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  10. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  11. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  12. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
6 more
  1. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  2. 2022CodeT: Code Generation with Generated Testscited by 369
  3. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 342
  4. 2018TextWorld: A Learning Environment for Text-based Gamescited by 304
  5. 2023Large Language Models Fail on Trivial Alterations to Theory-of-Mind Taskscited by 288
  6. 2022MultiPL-E: A Scalable and Extensible Approach to Benchmarking Neural Code Generationcited by 132

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 178; 2024: 543; 2025: 1010; 2026: 2333.

4,064 papers.

Sort:

By area:

  1. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  2. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  3. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  4. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  5. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  6. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
  7. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
  8. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 956
  9. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 814
  10. 2025AlphaEvolve: A coding agent for scientific and algorithmic discoverycited by 763
  11. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 752
  12. 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memorycited by 722
38 more
  1. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 677
  2. 2024A Survey on Large Language Models for Code Generationcited by 646
  3. 2023A Comprehensive Overview of Large Language Modelscited by 639
  4. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 622
  5. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 584
  6. 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviorscited by 580
  7. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 551
  8. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 527
  9. 2024Large Language Models: A Surveycited by 506
  10. 2025Group-in-Group Policy Optimization for LLM Agent Trainingcited by 442
  11. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 424
  12. 2023Cognitive Architectures for Language Agentscited by 422
  13. 2024Understanding the planning of LLM agents: A surveycited by 414
  14. 2024AFlow: Automating Agentic Workflow Generationcited by 395
  15. 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 390
  16. 2024TravelPlanner: A Benchmark for Real-World Planning with Language Agentscited by 385
  17. 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agentscited by 373
  18. 2024Automated Design of Agentic Systemscited by 360
  19. 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learningcited by 357
  20. 2025Agentic Context Engineering: Evolving Contexts for Self-Improving Language Modelscited by 349
  21. 2023MLAgentBench: Evaluating Language Agents on Machine Learning Experimentationcited by 339
  22. 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agentcited by 323
  23. 2024AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agentscited by 319
  24. 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 308
  25. 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMscited by 297
  26. 2024CRUXEval: A Benchmark for Code Reasoning, Understanding and Executioncited by 294
  27. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 292
  28. 2023SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planningcited by 289
  29. 2026SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Taskscited by 289
  30. 2026Reinforcement Learning via Self-Distillationcited by 285
  31. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  32. 2024R-Judge: Benchmarking Safety Risk Awareness for LLM Agentscited by 268
  33. 2024ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolutioncited by 266
  34. 2024Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Securitycited by 257
  35. 2023Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectivescited by 253
  36. 2026SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learningcited by 252
  37. 2024TextGrad: Automatic "Differentiation" via Textcited by 250
  38. 2023FireAct: Toward Language Agent Fine-tuningcited by 247

50 most cited of 4,064 All 4,064 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026ReflexiCoder: Teaching Large Language Models to Self-Reflect on Generated Code and Self-Correct It via Reinforcement Learning
  2. 2024MetaReflection: Learning Instructions for Language Agents using Past Reflections
  3. 2025SpeakRL: Synergizing Reasoning, Speaking, and Acting in Language Models with Reinforcement Learning
  4. 2023Is Feedback All You Need? Leveraging Natural Language Feedback in Goal-Conditioned Reinforcement Learning
  5. 2025PARL: Prompt-based Agents for Reinforcement Learning
  6. 2026Rethinking Agentic Reinforcement Learning In Large Language Models
  7. 2026From Reasoning to Agentic: Credit Assignment in Reinforcement Learning for Large Language Models
  8. 2026Reinforcement Inference: Leveraging Uncertainty for Self-Correcting Language Model Reasoning