mlsanityData to 7 Oct 2026

Runtime Burden Allocation for Structured LLM Routing in Agentic Expert Systems: A Full-Factorial Cross-Backend Methodology

arXiv 2604.01235 ↗First posted 26 Mar 2026cs.AIcited by 0

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Structured LLM routing is often treated as a prompt-engineering problem. We argue that it is, more fundamentally, a systems-level burden-allocation problem. As large language models (LLMs) become core control components in agentic AI systems, reliable structured routing must balance correctness, latency, and implementation cost under real deployment constraints. We show that this balance is shaped not only by prompts or schemas, but also by how structural work is allocated across the generation stack: whether output structure is emitted directly by the model, compressed during transport, or reconstructed locally after generation. We evaluate this formulation through a comprehensive full-factorial benchmark covering 48 deployment configurations and 15,552 requests across OpenAI, Gemini, and Llama backends. Our central finding is consequential: there is no universal best routing mode. Instead, backend-specific interaction effects dominate performance. Modes that remain highly reliable on Gemini and OpenAI can suffer substantial correctness degradation on Llama, while efficiency gains from compressed realization are strongly backend-dependent. Rather than presenting another isolated model comparison, this work contributes a deployable framework for reasoning about structured routing under heterogeneous backend conditions. We provide a cross-backend evaluation methodology and practical deployment guidance for navigating the correctness-cost-latency frontier in production-grade agentic expert systems.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 28 of its 35 references that are in the map. First posted on arXiv 26 Mar 2026.

Builds on

28 of 35 references.

Closest ancestors · 2

Same direction, newest first

  1. 2026ORCH: many analyses, one merge-a deterministic multi-agent orchestrator for discrete-choice reasoning with EMA-guided routing2 Feb
  2. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challenges21 Jan

Other references · 26

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  4. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  5. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  6. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  7. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  8. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  9. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
  10. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 441
  11. 2023LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Modelscited by 279
  12. 2024Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programscited by 270
14 more
  1. 2024RouterBench: A Benchmark for Multi-LLM Routing Systemcited by 222
  2. 2024AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?cited by 155
  3. 2023Large Language Model Cascades with Mixture of Thoughts Representations for Cost-efficient Reasoningcited by 140
  4. 2023Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Surveycited by 134
  5. 2024The Emerged Security and Privacy of LLM Agent: A Survey with Case Studiescited by 127
  6. 2024Reducing hallucination in structured outputs via Retrieval-Augmented Generationcited by 92
  7. 2024TinyAgent: Function Calling at the Edgecited by 55
  8. 2024Call Me When Necessary: LLMs can Efficiently and Faithfully Reason over Structured Environmentscited by 27
  9. 2024ATM: Adversarial Tuning Multi-agent System Makes a Robust Retrieval-Augmented Generatorcited by 19
  10. 2025SLOT: Structuring the Output of Large Language Modelscited by 18
  11. 2023Synergistic Integration of Large Language Models and Cognitive Architectures for Robust AI: An Exploratory Analysiscited by 11
  12. 2024Reasoning in Large Language Models: A Geometric Perspectivecited by 4
  13. 2025A Zero-Shot LLM Framework for Automatic Assignment Grading in Higher Educationcited by 3
  14. 2024Structured Object Language Modeling (SoLM): Native Structured Objects Generation Conforming to Complex Schemas with Self-Supervised Denoisingcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026EvoRoute: Experience-Driven Self-Routing LLM Agent Systems
  2. 2026Agentic Routing: The Harness-Native Data Flywheel
  3. 2024Designing Algorithms Empowered by Language Models: An Analytical Framework, Case Studies, and Insights
  4. 2026RSI-Router: Evolving Subtask-Level LLM Routing and Skills for Cost-Efficient Agents
  5. 2026Agent-as-a-Router: Agentic Model Routing for Coding Tasks
  6. 2026Training-Free Agentic AI: Probabilistic Control and Coordination in Multi-Agent LLM Systems
  7. 2026TwinRouterBench: Fast Static and Live Dynamic Evaluation for Realistic Agentic LLM Routing
  8. 2026AgentFloor: How Far Up the tool use Ladder Can Small Open-Weight Models Go?