mlsanityData to 7 Oct 2026

Large Language Model Agent: A Survey on Methodology, Applications and Challenges

arXiv 2503.21460 ↗First posted 27 Mar 2025cs.CLcited by 194

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

The era of intelligent agents is upon us, driven by revolutionary advancements in large language models. Large Language Model (LLM) agents, with goal-driven behaviors and dynamic adaptation capabilities, potentially represent a critical pathway toward artificial general intelligence. This survey systematically deconstructs LLM agent systems through a methodology-centered taxonomy, linking architectural foundations, collaboration mechanisms, and evolutionary pathways. We unify fragmented research threads by revealing fundamental connections between agent design principles and their emergent behaviors in complex environments. Our work provides a unified architectural perspective, examining how agents are constructed, how they collaborate, and how they evolve over time, while also addressing evaluation methodologies, tool applications, practical challenges, and diverse application domains. By surveying the latest developments in this rapidly evolving field, we offer researchers a structured taxonomy for understanding LLM agents and identify promising directions for future research. The collection is available at https://github.com/luo-junyu/Awesome-Agent-Papers.

Key facts. Cited by 194 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 255 of its 320 references that are in the map. First posted on arXiv 27 Mar 2025.

Builds on

255 of 320 references.

Closest ancestors · 34

Same direction, newest first

  1. 2024TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks18 Dec
  2. 2024Autonomous Industrial Control using an Agentic Framework with Large Language Models8 Nov
  3. 2024MMAU: A Holistic Benchmark of Agent Capabilities Across Diverse Domains18 Jul
  4. 2024GTA: A Benchmark for General Tool Agents11 Jul
  5. 2024AvaTaR: Optimizing LLM Agents for Tool Usage via Contrastive Reasoning17 Jun
  6. 2024Scaling Large Language Model-based Multi-Agent Collaboration11 Jun
  7. 2024Chain of Agents: Large Language Models Collaborating on Long-Context Tasks4 Jun
  8. 2024Towards Efficient LLM Grounding for Embodied Multi-Agent Collaboration23 May
  9. 2024Agent Planning with World Knowledge Model23 May
  10. 2024The Landscape of Emerging AI Agent Architectures for Reasoning, Planning, and Tool Calling: A Survey17 Apr
  11. 2024KnowAgent: Knowledge-Augmented Planning for LLM-Based Agents5 Mar
  12. 2024Dynamic Evaluation of Large Language Models by Meta Probing Agents21 Feb
22 more
  1. 2024Benchmark Self-Evolving: A Multi-Agent Framework for Dynamic LLM Evaluation18 Feb
  2. 2024Understanding the planning of LLM agents: A survey5 Feb
  3. 2024TravelPlanner: A Benchmark for Real-World Planning with Language Agents2 Feb
  4. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challenges21 Jan
  5. 2024Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Security10 Jan
  6. 2024Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospects7 Jan
  7. 2023LLM-MARS: Large Language Model for Behavior Tree Generation and NLP-enhanced Dialogue in Multi-Agent Robot Systems14 Dec
  8. 2023TrainerAgent: Customizable and Efficient Model Training through LLM-Powered Multi-Agent System11 Nov
  9. 2023OpenAgents: An Open Platform for Language Agents in the Wild16 Oct
  10. 2023A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaboration3 Oct
  11. 2023An In-depth Survey of Large Language Model-based Artificial Intelligence Agents23 Sep
  12. 2023The Rise and Potential of Large Language Model Based Agents: A Survey14 Sep
  13. 2023Cognitive Architectures for Language Agents5 Sep
  14. 2023A Survey on Large Language Model based Autonomous Agents22 Aug
  15. 2023ProAgent: Building Proactive Cooperative Agents with Large Language Models22 Aug
  16. 2023ExpeL: LLM Agents Are Experiential Learners20 Aug
  17. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate14 Aug
  18. 2023AgentBench: Evaluating LLMs as Agents7 Aug
  19. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework1 Aug
  20. 2023Inferring the Goals of Communicating Agents from Actions and Instructions28 Jun
  21. 2023AdaPlanner: Adaptive Planning from Feedback with Language Models26 May
  22. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society31 Mar

Other references · 221

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  7. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  8. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  9. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  10. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  11. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  12. 2020Extracting Training Data from Large Language Modelscited by 2,144
38 more
  1. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  2. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  3. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  4. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  5. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,411
  6. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
  7. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  8. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,210
  9. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
  10. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,002
  11. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
  12. 2021Ethical and social risks of harm from Language Modelscited by 957
  13. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
  14. 2023ChatDev: Communicative Agents for Software Developmentcited by 893
  15. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  16. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
  17. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  18. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
  19. 2023A Watermark for Large Language Modelscited by 667
  20. 2024Self-Rewarding Language Modelscited by 654
  21. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 622
  22. 2022Towards Reasoning in Large Language Models: A Surveycited by 593
  23. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  24. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 550
  25. 2024Executable Code Actions Elicit Better LLM Agentscited by 537
  26. 2022Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studiescited by 506
  27. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agentscited by 476
  28. 2024MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineeringcited by 429
  29. 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agentscited by 406
  30. 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Basescited by 402
  31. 2024AFlow: Automating Agentic Workflow Generationcited by 395
  32. 2024ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Searchcited by 338
  33. 2023MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoningcited by 311
  34. 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 309
  35. 2022Large Language Models are Better Reasoners with Self-Verificationcited by 304
  36. 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMscited by 297
  37. 2023Large Language Models as Tool Makerscited by 293
  38. 2023Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memorycited by 288

50 most cited of 221 All 221 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 91; 2026: 103.

194 papers.

Sort:

By area:

  1. 2025Memory in the Age of AI Agentscited by 285
  2. 2025Small Language Models are the Future of Agentic AIcited by 254
  3. 2025Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergycited by 204
  4. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 195
  5. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 155
  6. 2025A Survey of Context Engineering for Large Language Modelscited by 120
  7. 2026Agent Skills for Large Language Models: Architecture, Acquisition, Security, and the Path Forwardcited by 105
  8. 2026Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineeringcited by 68
  9. 2025TRiSM for Agentic AI: A Review of Trust, Risk, and Security Management in LLM-based Agentic Multi-Agent Systemscited by 58
  10. 2025From Web Search towards Agentic Deep Research: Incentivizing Search with Reasoning Agentscited by 49
  11. 2025AGENTIF: Benchmarking Instruction Following of Large Language Models in Agentic Scenarioscited by 44
  12. 2026A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agentscited by 40
38 more
  1. 2025Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AIcited by 39
  2. 2025Large Language Models for Planning: A Comprehensive and Systematic Surveycited by 38
  3. 2025A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontierscited by 34
  4. 2025From Language to Action: A Review of Large Language Models as Autonomous Agents and Tool Userscited by 32
  5. 2025A Survey on Autonomy-Induced Security Risks in Large Model-Based Agentscited by 28
  6. 2026OPID: On-Policy Skill Distillation for Agentic Reinforcement Learningcited by 28
  7. 2025FinMME: Benchmark Dataset for Financial Multi-Modal Reasoning Evaluationcited by 27
  8. 2025Free-MAD: Consensus-Free Multi-Agent Debatecited by 27
  9. 2025Building a Foundational Guardrail for General Agentic Systems via Synthetic Datacited by 26
  10. 2025LLM-based Agentic Reasoning Frameworks: A Survey from Methods to Scenarioscited by 25
  11. 2026Benchmark Test-Time Scaling of General LLM Agentscited by 24
  12. 2025Graphs Meet AI Agents: Taxonomy, Progress, and Future Opportunitiescited by 23
  13. 2026SEED: Self-Evolving On-Policy Distillation for Agentic Reinforcement Learningcited by 23
  14. 2025O2-Searcher: A Searching-based Agent Model for Open-Domain Open-Ended Question Answeringcited by 22
  15. 2026ReCreate: Reasoning and Creating Domain Agents Driven by Experiencecited by 22
  16. 2026OmniGAIA: Towards Native Omni-Modal AI Agentscited by 21
  17. 2025Multimodal DeepResearcher: Generating Text-Chart Interleaved Reports From Scratch with Agentic Frameworkcited by 20
  18. 2025AgentBound: Securing Execution Boundaries of AI Agentscited by 20
  19. 2025Adaptation of Agentic AI: A Survey of Post-Training, Memory, and Skillscited by 19
  20. 2026From Raw Experience to Skill Consumption: A Systematic Study of Model-Generated Agent Skillscited by 18
  21. 2025MME-Emotion: A Holistic Evaluation Benchmark for Emotional Intelligence in Multimodal Large Language Modelscited by 17
  22. 2025Emergent Misalignment via In-Context Learning: Narrow in-context examples can produce broadly misaligned LLMscited by 17
  23. 2025Multi-Agent Deep Research: Training Multi-Agent Systems with M-GRPOcited by 17
  24. 2025AgentScope 1.0: A Developer-Centric Framework for Building Agentic Applicationscited by 16
  25. 2025Agent Data Protocol: Unifying Datasets for Diverse, Effective Fine-tuning of LLM Agentscited by 16
  26. 2025Encouraging Good Processes Without the Need for Good Answers: Reinforcement Learning for LLM Agent Planningcited by 15
  27. 2026MemOCR: Layout-Aware Visual Memory for Efficient Long-Horizon Reasoningcited by 15
  28. 2026MemoryCD: Benchmarking Long-Context User Memory of LLM Agents for Lifelong Cross-Domain Personalizationcited by 15
  29. 2026From Skill Text to Skill Structure: The Scheduling-Structural-Logical Representation for Agent Skillscited by 15
  30. 2025The Future is Agentic: Definitions, Perspectives, and Open Challenges of Multi-Agent Recommender Systemscited by 14
  31. 2026Security Threat Modeling for Emerging AI-Agent Protocols: A Comparative Analysis of MCP, A2A, Agora, and ANPcited by 14
  32. 2026TurnOPD: Making On-Policy Distillation Turn-Aware for Efficient Long-Horizon Agent Trainingcited by 14
  33. 2026MiroFlow: Towards High-Performance and Robust Open-Source Agent Framework for General Deep Research Taskscited by 13
  34. 2025AgentGroupChat-V2: Divide-and-Conquer Is What LLM-Based Multi-Agent System Needcited by 12
  35. 2025SPAZER: Spatial-Semantic Progressive Reasoning Agent for Zero-shot 3D Visual Groundingcited by 12
  36. 2025Co-Sight: Enhancing LLM-Based Agents via Conflict-Aware Meta-Verification and Trustworthy Reasoning with Structured Factscited by 12
  37. 2025Runaway is Ashamed, But Helpful: On the Early-Exit Behavior of Large Language Model-based Agents in Embodied Environmentscited by 11
  38. 2025ToolACE-MT: Non-Autoregressive Generation for Agentic Multi-Turn Interactioncited by 11

50 most cited of 194 All 194 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Large Language Model-Based Agents for Software Engineering: A Survey
  2. 2024Practical Considerations for Agentic LLM Systems
  3. 2026Toward Formalizing LLM-Based Agent Designs through Structural Context Modeling and Semantic Dynamics Analysis
  4. 2024AgentGym: Evolving Large Language Model-based Agents across Diverse Environments
  5. 2025LLM-Agent-Controller: A Universal Multi-Agent Large Language Model System as a Control Engineer
  6. 2025Large Language Models Miss the Multi-Agent Mark
  7. 2024Designing Algorithms Empowered by Language Models: An Analytical Framework, Case Studies, and Insights
  8. 2025Generalizability of Large Language Model-Based Agents: A Comprehensive Survey