Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agents
On the map: LLM reasoning → Reasoning for agents, planning and code
also in 1 more
Abstract (arXiv)
With the rise of large language models (LLMs), LLM agents capable of autonomous reasoning, planning, and executing complex tasks have become a frontier in artificial intelligence. However, how to translate the research on general agents into productivity that drives industry transformations remains a significant challenge. To address this, this paper systematically reviews the technologies, applications, and evaluation methods of industry agents based on LLMs. Using an industry agent capability maturity framework, it outlines the evolution of agents in industry applications, from "process execution systems" to "adaptive social systems." First, we examine the three key technological pillars that support the advancement of agent capabilities: Memory, Planning, and Tool Use. We discuss how these technologies evolve from supporting simple tasks in their early forms to enabling complex autonomous systems and collective intelligence in more advanced forms. Then, we provide an overview of the application of industry agents in real-world domains such as digital engineering, scientific discovery, embodied intelligence, collaborative business execution, and complex system simulation. Additionally, this paper reviews the evaluation benchmarks and methods for both fundamental and specialized capabilities, identifying the challenges existing evaluation systems face regarding authenticity, safety, and industry specificity. Finally, we focus on the practical challenges faced by industry agents, exploring their capability boundaries, developmental potential, and governance issues in various scenarios, while providing insights into future directions. By combining technological evolution with industry practices, this review aims to clarify the current state and offer a clear roadmap and theoretical foundation for understanding and building the next generation of industry agents.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 391 of its 416 references that are in the map. First posted on arXiv 20 Oct 2025.
Builds on
391 of 416 references.
Closest ancestors · 9
Same direction, newest first
- 2025Introspection of Thought Helps AI Agents11 Jul
- 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systems12 Apr
- 2024AFlow: Automating Agentic Workflow Generation14 Oct
- 2024ACPBench: Reasoning about Action, Change, and Planning8 Oct
- 2024MapCoder: Multi-Agent Code Generation for Competitive Problem Solving18 May
- 2023Think-in-Memory: Recalling and Post-thinking Enable LLMs with Long-Term Memory15 Nov
- 2023Reasoning with Language Model is Planning with World Model24 May
- 2023CREATOR: Tool Creation for Disentangling Abstract and Concrete Reasoning of Large Language Models23 May
- 2023ART: Automatic multi-step reasoning and tool-use for large language models16 Mar
Other references · 382
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
- 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
- 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
38 more
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
- 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
- 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
- 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
- 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
- 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
- 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,210
- 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
- 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
- 2023GAIA: a benchmark for General AI Assistantscited by 1,084
- 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
- 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,002
- 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
- 2023ChatDev: Communicative Agents for Software Developmentcited by 893
- 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
- 2023ExpeL: LLM Agents Are Experiential Learnerscited by 814
- 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 800
- 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
- 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
- 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memorycited by 722
- 2023CogAgent: A Visual Language Model for GUI Agentscited by 692
- 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
- 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
- 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
- 2022PAL: Program-aided Language Modelscited by 635
- 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 622
50 most cited of 382 All 382 (JSON)
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025Agentic Systems: A Guide to Transforming Industries with Vertical AI Agents
- 2025LLMs Working in Harmony: A Survey on the Technological Aspects of Building Effective LLM-Based Multi Agent Systems
- 2025Survey on Evaluation of LLM-based Agents
- 2024Practical Considerations for Agentic LLM Systems
- 2025Evaluation and Benchmarking of LLM Agents: A Survey
- 2025LLM-Powered AI Agent Systems and Their Applications in Industry
- 2026Developing LLM-based Multi-Agent Systems in Software Engineering: A Mixed-Method Experience Report
- 2023The Rise and Potential of Large Language Model Based Agents: A Survey