mlsanityData to 14 Oct 2026

LLM-Powered AI Agent Systems and Their Applications in Industry

arXiv 2505.16120 ↗First posted 22 May 2025cs.AIcited by 22

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

The emergence of Large Language Models (LLMs) has reshaped agent systems. Unlike traditional rule-based agents with limited task scope, LLM-powered agents offer greater flexibility, cross-domain reasoning, and natural language interaction. Moreover, with the integration of multi-modal LLMs, current agent systems are highly capable of processing diverse data modalities, including text, images, audio, and structured tabular data, enabling richer and more adaptive real-world behavior. This paper comprehensively examines the evolution of agent systems from the pre-LLM era to current LLM-powered architectures. We categorize agent systems into software-based, physical, and adaptive hybrid systems, highlighting applications across customer service, software development, manufacturing automation, personalized education, financial trading, and healthcare. We further discuss the primary challenges posed by LLM-powered agents, including high inference latency, output uncertainty, lack of evaluation metrics, and security vulnerabilities, and propose potential solutions to mitigate these concerns.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 77 of its 123 references that are in the map. First posted on arXiv 22 May 2025.

Builds on

77 of 123 references.

Closest ancestors · 8

Same direction, newest first

  1. 2024Deploying Foundation Model Powered Agent Services: A Survey18 Dec
  2. 2024A Large Language Model-based multi-agent manufacturing system for intelligent shopfloor27 May
  3. 2024The Landscape of Emerging AI Agent Architectures for Reasoning, Planning, and Tool Calling: A Survey17 Apr
  4. 2024Large Multimodal Agents: A Survey23 Feb
  5. 2024Understanding the planning of LLM agents: A survey5 Feb
  6. 2024TravelPlanner: A Benchmark for Real-World Planning with Language Agents2 Feb
  7. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challenges21 Jan
  8. 2024Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Security10 Jan

Other references · 69

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  3. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  4. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  5. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  6. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  7. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  8. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  9. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  10. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  11. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  12. 2023GAIA: a benchmark for General AI Assistantscited by 1,128
38 more
  1. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  2. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 900
  3. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 800
  4. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 737
  5. 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 650
  6. 2023A Comprehensive Overview of Large Language Modelscited by 648
  7. 2023ChemCrow: Augmenting large-language models with chemistry toolscited by 631
  8. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 575
  9. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 526
  10. 2017A Brief Survey of Deep Reinforcement Learningcited by 483
  11. 2023LLM-QAT: Data-Free Quantization Aware Training for Large Language Modelscited by 324
  12. 2023Certifying LLM Safety against Adversarial Promptingcited by 266
  13. 2024AI Agents That Mattercited by 219
  14. 2024WebLINX: Real-World Website Navigation with Multi-Turn Dialoguecited by 180
  15. 2018A Survey and Critique of Multiagent Deep Reinforcement Learningcited by 177
  16. 2023LLM in a flash: Efficient Large Language Model Inference with Limited Memorycited by 177
  17. 2024Agent AI: Surveying the Horizons of Multimodal Interactioncited by 170
  18. 2024MDAgents: An Adaptive Collaboration of LLMs for Medical Decision-Makingcited by 163
  19. 2024Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fastcited by 153
  20. 2023GPT-4V in Wonderland: Large Multimodal Models for Zero-Shot Smartphone GUI Navigationcited by 137
  21. 2024A Multimodal Foundation Agent for Financial Trading: Tool-Augmented, Diversified, and Generalistcited by 137
  22. 2024The Emerged Security and Privacy of LLM Agent: A Survey with Case Studiescited by 131
  23. 2023Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Modelscited by 129
  24. 2018Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applicationscited by 127
  25. 2023SmartPlay: A Benchmark for LLMs as Intelligent Agentscited by 125
  26. 2024A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directionscited by 121
  27. 2024Efficient Adversarial Training in LLMs with Continuous Attackscited by 113
  28. 2024From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Futurecited by 86
  29. 2025Harnessing Multiple Large Language Models: A Survey on LLM Ensemblecited by 86
  30. 2025A Survey of LLM-based Agents in Medicine: How far are we from Baymax?cited by 85
  31. 2024Safeguarding Large Language Models: A Surveycited by 78
  32. 2024A Complete Survey on LLM-based AI Chatbotscited by 71
  33. 2024Building Guardrails for Large Language Modelscited by 65
  34. 2024FlightLLM: Efficient Large Language Model Inference with a Complete Mapping Flow on FPGAscited by 64
  35. 2024LLM With Tools: A Surveycited by 61
  36. 2024Large Language Model Agent in Financial Trading: A Surveycited by 60
  37. 2025LLM Agents for Education: Advances and Applicationscited by 59
  38. 2023Baby Llama: knowledge distillation from an ensemble of teachers trained on a small dataset with no performance penaltycited by 57

50 most cited of 69 All 69 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 12; 2026: 10.

22 papers.

Sort:

By area:

  1. 2025A Survey of Context Engineering for Large Language Modelscited by 126
  2. 2025AgentOrchestra: Orchestrating Multi-Agent Intelligence with the Tool-Environment-Agent(TEA) Protocolcited by 39
  3. 2025Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecyclecited by 35
  4. 2026Toward Personalized LLM-Powered Agents: Foundations, Evaluation, and Future Directionscited by 14
  5. 2025Tapas Are Free! Training-Free Adaptation of Programmatic Agents via LLM-Guided Program Synthesis in Dynamic Environmentscited by 9
  6. 2025PARSE: LLM Driven Schema Optimization for Reliable Entity Extractioncited by 6
  7. 2026AMA: Adaptive Memory via Multi-Agent Collaborationcited by 6
  8. 2025A Survey of Foundation Models for IoT: Taxonomy and Criteria-Based Analysiscited by 5
  9. 2025Beyond Formal Semantics for Capabilities and Skills: Model Context Protocol in Manufacturingcited by 4
  10. 2025Agentic AI for autonomous anomaly management in complex systemscited by 4
  11. 2025Reasoning over Boundaries: Enhancing Specification Alignment via Test-time Deliberationcited by 4
  12. 2026FAMA: Failure-Aware Meta-Agentic Framework for Open-Source LLMs in Interactive Tool Use Environmentscited by 4
10 more
  1. 2026LLM-Based Multi-Agent Systems for Code Generation: A Multi-Vocal Literature Reviewcited by 3
  2. 2025Mina: A Multilingual LLM-Powered Legal Assistant Agent for Bangladesh for Empowering Access to Justicecited by 2
  3. 2025Lumos: Let there be Language Model System Certificationcited by 1
  4. 2025Mathematical Framing for Different Agent Strategiescited by 1
  5. 2026Foundation-Model-Based Agents in Industrial Automation: Purposes, Capabilities, and Open Challengescited by 1
  6. 2026Benchmarking LLM Serving Systems for Agentic AI Workloads with XPerfcited by 1
  7. 2026AI Agent Systems for Supply Chains: Structured Decision Prompts and Memory Retrievalcited by 0
  8. 2026Structure-Aware RAG: Structured Retrieval Augmented Generation from Noisy Data for Conversational Agentscited by 0
  9. 2026MESA: Prioritizing Vulnerable Communication Channels for Securing Multi-Agent Systemscited by 0
  10. 2026A Safety-Bounded SDC-to-MCP Gateway for Medical AI Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospects
  2. 2023The Rise and Potential of Large Language Model Based Agents: A Survey
  3. 2023A Survey on Large Language Model based Autonomous Agents
  4. 2024Practical Considerations for Agentic LLM Systems
  5. 2023Balancing Autonomy and Alignment: A Multi-Dimensional Taxonomy for Autonomous LLM-powered Multi-Agent Architectures
  6. 2024Exploring Autonomous Agents through the Lens of Large Language Models: A Review
  7. 2025Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agents
  8. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challenges