mlsanityData to 14 Oct 2026

In-Context Prompting Obsoletes Agent Orchestration for Procedural Tasks

arXiv 2604.27891 ↗First posted 30 Apr 2026cs.AI cs.LGcited by 2

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Agent orchestration frameworks -- LangGraph, CrewAI, Google ADK, OpenAI Agents SDK, and others -- place an external orchestrator above the LLM, tracking state and injecting routing instructions at every turn. We present a controlled comparison showing that for procedural tasks, this architecture is dominated by a simpler alternative: putting the entire procedure in the system prompt and letting the model self-orchestrate. Across three domains -- travel booking (14 nodes), Zoom technical support (14 nodes), and insurance claims processing (55 nodes) -- we evaluate 200 conversations per condition using LLM-as-judge scoring on five quality criteria. The in-context approach scores 4.53--5.00 on a 5-point scale while a LangGraph orchestrator using the same model scores 4.17--4.84. The orchestrated system fails on 24% of travel, 9% of Zoom, and 17% of insurance conversations, compared to 11.5%, 0.5%, and 5% for the in-context baseline. While external orchestration may have been necessary for earlier models, advances in frontier model capabilities have made it unnecessary for multi-turn conversations following a defined procedure.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 21 of its 30 references that are in the map. First posted on arXiv 30 Apr 2026.

Builds on

21 of 30 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026When Single-Agent with Skills Replace Multi-Agent Systems and When They Fail8 Jan
  2. 2025Towards a Science of Scaling Agent Systems9 Dec
  3. 2025Procedural Knowledge Improves Agentic LLM Workflows10 Nov
  4. 2024Agent-FLAN: Designing Data and Methods of Effective Agent Tuning for Large Language Models19 Mar
  5. 2023FireAct: Toward Language Agent Fine-tuning9 Oct
  6. 2023AgentBench: Evaluating LLMs as Agents7 Aug

Other references · 15

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  4. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  5. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 766
  6. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 737
  7. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
  8. 2025Why Do Multi-Agent LLM Systems Fail?cited by 604
  9. 2020A Simple Language Model for Task-Oriented Dialoguecited by 332
  10. 2024AI Agents That Mattercited by 219
  11. 2026ReliabilityBench: Evaluating LLM Agent Reliability Under Production-Like Stress Conditionscited by 36
  12. 2023Large Language Models can accomplish Business Process Management Taskscited by 34
3 more
  1. 2024WorkflowLLM: Enhancing Workflow Orchestration Capability of Large Language Modelscited by 30
  2. 2025An Empirical Study of Agent Developer Practices in AI Agent Frameworkscited by 14
  3. 2024Simulating Task-Oriented Dialogues with State Transition Graphs and Large Language Modelscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Compiling Agentic Workflows into LLM Weights: Near-Frontier Quality at Two Orders of Magnitude Less Costcited by 5
  2. 2026StateM: Reaching 95.3% Raw Accuracy, or a 15 Frontier Run, on Terminal-Bench 2.1 via Harness Scalingcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026PerspectiveGap: A Benchmark for Multi-Agent Orchestration Prompting
  2. 2026Utility-Guided Agent Orchestration for Efficient LLM Tool Use
  3. 2026Empirical Comparison of Agent Communication Protocols for Task Orchestration
  4. 2026Less Context, Better Agents: Efficient Context Engineering for Long-Horizon Tool-Using LLM Agents
  5. 2025When Refusals Fail: Unstable Safety Mechanisms in Long-Context LLM Agents
  6. 2026Dynamic Attentional Context Scoping: Agent-Triggered Focus Sessions for Isolated Per-Agent Steering in Multi-Agent LLM Orchestration
  7. 2025Multi-Agent LLM Orchestration Achieves Deterministic, High-Quality Decision Support for Incident Response
  8. 2025Plan-Then-Execute: An Empirical Study of User Trust and Team Performance When Using LLM Agents As A Daily Assistant