mlsanityData to 7 Oct 2026

Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agents

arXiv 2510.17491 ↗First posted 20 Oct 2025cs.CLcited by 2

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

With the rise of large language models (LLMs), LLM agents capable of autonomous reasoning, planning, and executing complex tasks have become a frontier in artificial intelligence. However, how to translate the research on general agents into productivity that drives industry transformations remains a significant challenge. To address this, this paper systematically reviews the technologies, applications, and evaluation methods of industry agents based on LLMs. Using an industry agent capability maturity framework, it outlines the evolution of agents in industry applications, from "process execution systems" to "adaptive social systems." First, we examine the three key technological pillars that support the advancement of agent capabilities: Memory, Planning, and Tool Use. We discuss how these technologies evolve from supporting simple tasks in their early forms to enabling complex autonomous systems and collective intelligence in more advanced forms. Then, we provide an overview of the application of industry agents in real-world domains such as digital engineering, scientific discovery, embodied intelligence, collaborative business execution, and complex system simulation. Additionally, this paper reviews the evaluation benchmarks and methods for both fundamental and specialized capabilities, identifying the challenges existing evaluation systems face regarding authenticity, safety, and industry specificity. Finally, we focus on the practical challenges faced by industry agents, exploring their capability boundaries, developmental potential, and governance issues in various scenarios, while providing insights into future directions. By combining technological evolution with industry practices, this review aims to clarify the current state and offer a clear roadmap and theoretical foundation for understanding and building the next generation of industry agents.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 391 of its 416 references that are in the map. First posted on arXiv 20 Oct 2025.

Builds on

391 of 416 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025Introspection of Thought Helps AI Agents11 Jul
  2. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systems12 Apr
  3. 2024AFlow: Automating Agentic Workflow Generation14 Oct
  4. 2024ACPBench: Reasoning about Action, Change, and Planning8 Oct
  5. 2024MapCoder: Multi-Agent Code Generation for Competitive Problem Solving18 May
  6. 2023Think-in-Memory: Recalling and Post-thinking Enable LLMs with Long-Term Memory15 Nov
  7. 2023Reasoning with Language Model is Planning with World Model24 May
  8. 2023CREATOR: Tool Creation for Disentangling Abstract and Concrete Reasoning of Large Language Models23 May
  9. 2023ART: Automatic multi-step reasoning and tool-use for large language models16 Mar

Other references · 382

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  8. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  9. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  10. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  11. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  12. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
38 more
  1. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  2. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  3. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  4. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  5. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  6. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  7. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  8. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  9. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  10. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  11. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  12. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  13. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  14. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  15. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  16. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
  17. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
  18. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  19. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,210
  20. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  21. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  22. 2023GAIA: a benchmark for General AI Assistantscited by 1,084
  23. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  24. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,002
  25. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  26. 2023ChatDev: Communicative Agents for Software Developmentcited by 893
  27. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  28. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 814
  29. 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 800
  30. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
  31. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  32. 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memorycited by 722
  33. 2023CogAgent: A Visual Language Model for GUI Agentscited by 692
  34. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
  35. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
  36. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  37. 2022PAL: Program-aided Language Modelscited by 635
  38. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 622

50 most cited of 382 All 382 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Agentic Systems: A Guide to Transforming Industries with Vertical AI Agents
  2. 2025LLMs Working in Harmony: A Survey on the Technological Aspects of Building Effective LLM-Based Multi Agent Systems
  3. 2025Survey on Evaluation of LLM-based Agents
  4. 2024Practical Considerations for Agentic LLM Systems
  5. 2025Evaluation and Benchmarking of LLM Agents: A Survey
  6. 2025LLM-Powered AI Agent Systems and Their Applications in Industry
  7. 2026Developing LLM-based Multi-Agent Systems in Software Engineering: A Mixed-Method Experience Report
  8. 2023The Rise and Potential of Large Language Model Based Agents: A Survey