From Inference Routing to Agent Orchestration: Declarative Policy Compilation with Cross-Layer Verification
On the map: LLM agents and tool use → Agent threats, guardrails and governance
also in 1 more
Abstract (arXiv)
The Semantic Router DSL is a non-Turing-complete policy language deployed in production for per-request LLM inference routing: content signals (embedding similarity, PII detection, jailbreak scoring) feed into weighted projections and priority-ordered decision trees that select a model, enforce privacy policies, and produce structured audit traces -- all from a single declarative source file. Prior work established conflict-free compilation for probabilistic predicates and positioned the DSL within the Workload-Router-Pool inference architecture. This paper extends the same language from stateless, per-request routing to multi-step agent workflows -- the full path from inference gateway to agent orchestration to infrastructure deployment. The DSL compiler emits verified decision nodes for orchestration frameworks (LangGraph, OpenClaw), Kubernetes artifacts (NetworkPolicy, Sandbox CRD, ConfigMap), YANG/NETCONF payloads, and protocol-boundary gates (MCP, A2A) -- all from the same source. Because the language is non-Turing-complete, the compiler guarantees exhaustive routing, conflict-free branching, referential integrity, and audit traces structurally coupled to the decision logic. Because signal definitions are shared across targets, a threshold change propagates from inference gateway to agent gate to infrastructure artifact in one compilation step -- eliminating cross-team coordination as the primary source of policy drift. We ground the approach in four pillars -- auditability, cost efficiency, verifiability, and tunability -- and identify the verification boundary at each layer.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 7 of its 33 references that are in the map. First posted on arXiv 28 Mar 2026.
Builds on
7 of 33 references.
References · 7
- 2017On Calibration of Modern Neural Networkscited by 4,860
- 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 791
- 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 633
- 2024Hybrid LLM: Cost-Efficient and Quality-Aware Query Routingcited by 354
- 2024A Unified Approach to Routing and Cascading for LLMscited by 71
- 2026Conflict-Free Policy Languages for Probabilistic ML Predicates: A Framework and Case Study with the Semantic Router DSLcited by 2
- 2026The Workload-Router-Pool Architecture for LLM Inference Optimization: A Vision Paper from the vLLM Semantic Router Projectcited by 1
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026AgentFlow: A Flow-Centric Policy Language and Framework for Securing LLM Agent Systems
- 2026Formal Verification of Agentic Systems over Operational Data
- 2026Context Kubernetes: Declarative Orchestration of Enterprise Knowledge for Agentic AI Systems
- 2026LOGIGEN: Logic-Driven Generation of Verifiable Agentic Tasks
- 2026From Natural Language Policies to Executable Obligations: A Verification Harness for Dependable In-Car LLM Agents
- 2026NetInspector: Measuring and Improving LLM Capabilities for Reliable Intent-Based Networking Policy Generation ↗
- 2026Uno-Orchestra: Parsimonious Agent Routing via Selective Delegation
- 2026Can Agent Harnesses and Inference Engines Hear Each Other? The HEAR Protocol for Agentic LLM Serving