Agents in the Wild: Where Research Meets Deployment
On the map: LLM reasoning → Reasoning for agents, planning and code
also in 1 more
Abstract (arXiv)
Agentic systems large language model (LLM) based architectures capable of reasoning, planning, acting, and coordinating with tools and other agents are rapidly transitioning from research prototypes to production scale deployments across domains such as software engineering, scientific discovery, and finance. While academic work has emphasized benchmarks and algorithmic innovation, deployment raises new challenges around robustness, safety, and reliability. This tutorial brings together researchers and practitioners to explore advances in reasoning and planning, multi agent coordination, and evaluation, highlighting open challenges arising from deployment experience. Through applied case studies in pharmaceutical discovery and financial systems, we analyze common design patterns that make agentic systems successful, and discuss practical mitigation strategies for failure modes, such as verification pipelines, fallback mechanisms, and human in the loop supervision. Attendees will gain a comprehensive view of the field along with concrete design patterns, evaluation checklists, and templates for safe and reliable deployment across industries.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 48 of its 70 references that are in the map. First posted on arXiv 21 Jul 2026.
Builds on
48 of 70 references.
Closest ancestors · 5
Same direction, newest first
- 2025LLM-based Agentic Reasoning Frameworks: A Survey from Methods to Scenarios25 Aug
- 2025MultiAgentBench: Evaluating the Collaboration and Competition of LLM agents3 Mar
- 2025Agent-R: Training Language Model Agents to Reflect via Iterative Self-Training20 Jan
- 2024WorkArena++: Towards Compositional Planning and Reasoning-based Common Knowledge Work Tasks7 Jul
- 2024Tool-Planner: Task Planning with Clusters across Multiple Tools6 Jun
Other references · 43
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
- 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
- 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 923
- 2023ChemCrow: Augmenting large-language models with chemistry toolscited by 631
- 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 431
- 2024Understanding the planning of LLM agents: A surveycited by 421
- 2025Accelerating scientific discovery with Co-Scientistcited by 401
- 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agentscited by 369
31 more
- 2024WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?cited by 363
- 2024Agent-SafetyBench: Evaluating the Safety of LLM Agentscited by 306
- 2024ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discoverycited by 235
- 2024Magentic-One: A Generalist Multi-Agent System for Solving Complex Taskscited by 232
- 2024Tool Learning with Large Language Models: A Surveycited by 223
- 2025Survey on Evaluation of LLM-based Agentscited by 204
- 2023FinMem: A Performance-Enhanced LLM Trading Agent with Layered Memory and Character Designcited by 164
- 2024The BrowserGym Ecosystem for Web Agent Researchcited by 161
- 2025Evaluation and Benchmarking of LLM Agents: A Surveycited by 152
- 2024A Multimodal Foundation Agent for Financial Trading: Tool-Augmented, Diversified, and Generalistcited by 137
- 2024ST-WebAgentBench: A Benchmark for Evaluating Safety and Trustworthiness in Web Agentscited by 133
- 2024Empowering Biomedical Discovery with AI Agentscited by 118
- 2024SciAgents: Automating scientific discovery through multi-agent intelligent graph reasoningcited by 94
- 2025A Survey of WebAgents: Towards Next-Generation AI Agents for Web Automation with Large Foundation Modelscited by 87
- 2025PlanGenLLMs: A Modern Survey of LLM Planning Capabilitiescited by 83
- 2024Towards Lifelong Learning of Large Language Models: A Surveycited by 64
- 2025Robin: A multi-agent system for automating scientific discoverycited by 62
- 2025Training a Scientific Reasoning Model for Chemistrycited by 56
- 2024TDAG: A Multi-Agent Framework based on Dynamic Task Decomposition and Agent Generationcited by 51
- 2024MobileSafetyBench: Evaluating Safety of Autonomous Agents in Mobile Device Controlcited by 51
- 2025CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactionscited by 47
- 2025Magentic-UI: Towards Human-in-the-loop Agentic Systemscited by 43
- 2025AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructionscited by 38
- 2024Multi-Agent Collaboration via Cross-Team Orchestrationcited by 30
- 2024Planning with Multi-Constraints via Collaborative Language Agentscited by 26
- 2024Advancing Agentic Systems: Dynamic Task Decomposition, Tool Integration and Evaluation using Novel Metrics and Datasetcited by 24
- 2025PharmAgents: Building a Virtual Pharma with Large Language Model Agentscited by 21
- 2024AutoGen Studio: A No-Code Developer Tool for Building and Debugging Multi-Agent Systemscited by 20
- 2024Search Engines in an AI Era: The False Promise of Factual and Verifiable Source-Cited Responsescited by 15
- 2025DeepTRACE: Auditing Deep Research AI Systems for Tracking Reliability Across Citations and Evidencecited by 10
- 2025Beyond the Strongest LLM: Multi-Turn Multi-Agent Orchestration vs. Single LLMs on Benchmarkscited by 9
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Architecting Agentic Communities using Design Patterns
- 2026LLM-Enabled Multi-Agent Systems: Empirical Evaluation and Insights into Emerging Design Patterns & Paradigms
- 2024Practical Considerations for Agentic LLM Systems
- 2025Risk Analysis Techniques for Governed LLM-based Multi-Agent Systems
- 2026LLM-Based Agents for Software and Systems Security: Approaches, Applications, and Assessment
- 2026ORAgentBench: Can LLM Agents Solve Challenging Operations Research Tasks End to End?
- 2025Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agents
- 2026LLM-Based Agentic Systems for Software Engineering: Challenges and Opportunities