mlsanityData to 7 Oct 2026

Can Agent Harnesses and Inference Engines Hear Each Other? The HEAR Protocol for Agentic LLM Serving

arXiv 2610.06597 ↗First posted 5 Oct 2026cs.AIcited by 0

On the map: LLM agents and tool use → Software engineering coding agents

also in 1 more
Abstract (arXiv)

LLM agents increasingly execute complex workflows involving multi-turn reasoning, tool use, and parallel agents. Efficient serving requires decisions that span two layers with complementary information: the agent harness understands workflow dependencies, context lifecycles, and execution objectives, whereas the inference engine observes request queues, KV-cache state, resource pressure, and execution capabilities. Existing interfaces do not systematically connect these views, limiting workflow-aware execution. HEAR, a bidirectional Harness--Engine Pairing protocol for agentic LLM serving. HEAR standardizes how the harness communicates workflow intent and execution requirements and how the engine returns runtime state, capabilities, and outcomes. By separating protocol semantics from optimization policies, HEAR supports diverse coordination strategies without changing workflow or model semantics. We instantiate HEAR for online cache-aware runtime coordination and workload-aware execution-mode selection for agent roles. Across four conversational and research-agent benchmarks under memory-constrained, concurrent serving, HEAR achieves a 1.61× batch speedup and reduces median time-to-first-token by 2.23× on SCBench. Mooncake shows that workflow intent and live engine state provide complementary benefits across load regimes. On BrowseComp-Plus and DeepResearchBench, workload-specific configurations yield 1.23× and 2.45× end-to-end speedups, respectively, without observed task-quality degradation. These results establish HEAR as a reusable coordination substrate for efficient agentic LLM serving.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 24 of its 33 references that are in the map. First posted on arXiv 5 Oct 2026.

Builds on

24 of 33 references.

Closest ancestors · 8

Same direction, newest first

  1. 2026A Policy-Driven Runtime Layer for Agentic LLM Serving26 May
  2. 2026Efficient Serving for Dynamic Agent Workflows with Prediction-based KV-Cache Management7 May
  3. 2026Pythia: Exploiting Workflow Predictability for Efficient Agent-Native LLM Serving28 Apr
  4. 2026Efficient LLM Serving for Agentic Workflows: A Data Systems Perspective17 Mar
  5. 2025Continuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Live4 Nov
  6. 2025Pie: A Programmable Serving System for Emerging LLM Applications28 Oct
  7. 2025KVFlow: Efficient Prefix Caching for Accelerating LLM-Based Multi-Agent Workflows10 Jul
  8. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation16 Aug

Other references · 16

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  3. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  4. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 805
  5. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 604
  6. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Servecited by 507
  7. 2025DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agentscited by 236
  8. 2024Mooncake: A KVCache-centric Disaggregated Architecture for LLM Servingcited by 193
  9. 2025LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inferencecited by 119
  10. 2025Multi-Agent Collaboration via Evolving Orchestrationcited by 102
  11. 2024Parrot: Efficient Serving of LLM-based Applications with Semantic Variablecited by 96
  12. 2024SCBench: A KV Cache-Centric Analysis of Long-Context Methodscited by 59
4 more
  1. 2024InferCept: Efficient Intercept Support for Augmented Large Language Model Inferencecited by 32
  2. 2025Learning to Orchestrate Agents in Natural Language with the Conductorcited by 22
  3. 2025AgentScope 1.0: A Developer-Centric Framework for Building Agentic Applicationscited by 16
  4. 2026CONCUR: High-Throughput Agentic Batch Inference of LLM via Congestion-Based Concurrency Controlcited by 13

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Sutradhara: An Intelligent Orchestrator-Engine Co-design for Tool-based Agentic Inference
  2. 2026Harness-Bench: Measuring Harness Effects across Models in Realistic Agent Workflows
  3. 2026Substrate-Portable Execution for Production LLM Workflows
  4. 2026GraphBit: A Graph-based Agentic Framework for Non-Linear Agent Orchestration
  5. 2026ThunderAgent: A Simple, Fast and Program-Aware Agentic Inference System
  6. 2026HexAGenT: Efficient Agentic LLM Serving via Workflow- and Heterogeneity-Aware Scheduling
  7. 2026Harnesses for Inference-Time Alignment over Execution Trajectories
  8. 2026Code as Agent Harness