mlsanityData to 7 Oct 2026

DynBranch: Speculative Subgraph Reuse for Dynamic Agentic LLM Serving

arXiv 2609.31047 ↗First posted 25 Sep 2026cs.DC cs.AI cs.LGcited by 0

On the map: LLM agents and tool use → Software engineering coding agents

also in 1 more
Abstract (arXiv)

Agentic LLM workflows decide their execution paths at runtime. Downstream computation may be predictable, or may have run before, yet it cannot begin until the model or the user resolves the branch. We call this serialization the branch-resolution barrier. Caching alone does not hide it: the key that identifies a reusable result is not known until then. In this paper, we propose DynBranch, which makes an unresolved branch addressable before it resolves. Its stable coordinate lets candidate subgraphs run during resolution and completed subgraph results be reused across later requests. A two-level controller admits this work when its expected benefit exceeds the load price. DynBranch sits at the model-API boundary and requires no changes to agent harnesses or model execution engines. Across four agentic workloads with Qwen3-32B on 4x H200 GPUs, DynBranch reduces mean latency by up to 32% over each workload's strongest prior system and by 46-66% against a no-reuse floor, while preserving workflow results. The benefit persists across backbone families and on a commodity Qwen3-8B/RTX 4090 deployment.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 54 of its 112 references that are in the map. First posted on arXiv 25 Sep 2026.

Builds on

54 of 112 references.

Closest ancestors · 15

Same direction, newest first

  1. 2026From LLM Inference to Agentic Workloads: Characterization and Implications for Serving Systems15 Aug
  2. 2026Speculate with Memory: Lossless Acceleration for LLM Agents14 Jul
  3. 2026SPORK: Self-Speculative Forking to Accelerate Agentic LLM Inference3 Jul
  4. 2026TraceLab: Characterizing Coding Agent Workloads for LLM Serving29 Jun
  5. 2026A Policy-Driven Runtime Layer for Agentic LLM Serving26 May
  6. 2026B-PASTE: Beam-Aware Pattern-Guided Speculative Execution for Resource-Constrained LLM Agents9 Apr
  7. 2026Parallelizing Tool Execution and LLM Generation for Low-Latency Agent Serving19 Mar
  8. 2026Efficient LLM Serving for Agentic Workflows: A Data Systems Perspective17 Mar
  9. 2025Reducing Latency of LLM Search Agent via Speculation-based Algorithm-System Co-Design25 Nov
  10. 2025Continuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Live4 Nov
  11. 2025Understanding Code Agent Behaviour: An Empirical Study of Success and Failure Trajectories31 Oct
  12. 2025Speculative Actions: A Lossless Framework for Faster Agentic Systems5 Oct
3 more
  1. 2025KVFlow: Efficient Prefix Caching for Accelerating LLM-Based Multi-Agent Workflows10 Jul
  2. 2025Agentic Plan Caching: Test-Time Memory for Fast and Cost-Efficient LLM Agents17 Jun
  3. 2025SWE-rebench: An Automated Pipeline for Task Collection and Decontaminated Evaluation of Software Engineering Agents26 May

Other references · 39

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  4. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  5. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  6. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  7. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
  8. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 821
  9. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 633
  10. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Servecited by 507
  11. 2019Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Datasetcited by 426
  12. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verificationcited by 359
27 more
  1. 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusioncited by 211
  2. 2024AgentScope: A Flexible yet Robust Multi-Agent Platformcited by 120
  3. 2024Parrot: Efficient Serving of LLM-based Applications with Semantic Variablecited by 96
  4. 2024Speculative RAG: Enhancing Retrieval Augmented Generation through Draftingcited by 76
  5. 2024Preble: Efficient Distributed Prompt Scheduling for LLM Servingcited by 68
  6. 2018Linear Bandits with Stochastic Delayed Feedbackcited by 55
  7. 2025KVCache Cache in the Wild: Characterizing and Optimizing KVCache Cache at a Large Cloud Providercited by 41
  8. 2024TurboSpec: Closed-loop Speculation Control System for Optimizing LLM Serving Goodputcited by 36
  9. 2025Cache-Craft: Managing Chunk-Caches for Efficient Retrieval-Augmented Generationcited by 36
  10. 2024Teola: Towards End-to-End Optimization of LLM-based Applicationscited by 32
  11. 2025Mind the Memory Gap: Unveiling GPU Bottlenecks in Large-Batch LLM Inferencecited by 32
  12. 2024GPT Semantic Cache: Reducing LLM Costs and Latency via Semantic Embedding Cachingcited by 21
  13. 2025Dynamic Speculative Agent Planningcited by 10
  14. 2024Alto: Orchestrating Distributed Compound AI Systems with Nested Ancestrycited by 7
  15. 2026ROSE: Rollout On Serving GPUs via Cooperative Elasticity for Agentic RLcited by 7
  16. 2024InstCache: A Predictive Cache for LLM Servingcited by 6
  17. 2025Batch Query Processing and Optimization for Agentic Workflowscited by 6
  18. 2025FlowMesh: A Service Fabric for Composable LLM Workflowscited by 6
  19. 2025Nightjar: Dynamic Adaptive Speculative Decoding for Large Language Models Servingcited by 6
  20. 2026DualSpec: Accelerating Deep Research Agents via Dual-Process Action Speculationcited by 6
  21. 2017Engineering Record And Replay For Deployability: Extended Technical Reportcited by 5
  22. 2025Cortex: Achieving Low-Latency, Cost-Efficient Remote Data Access For LLM via Semantic-Aware Knowledge Cachingcited by 4
  23. 2026IdleSpec: Exploiting Idle Time via Speculative Planning for LLM Agentscited by 4
  24. 2013Low latency via redundancycited by 2
  25. 2025Semi-Clairvoyant Scheduling of Speculative Decoding Requests to Minimize LLM Inference Latencycited by 2
  26. 2025Dialogue Response Prefetching Based on Semantic Similarity and Prediction Confidence of Language Modelcited by 2
  27. 2026Skim: Speculative Execution for Fast and Efficient Web Agentscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Serving Agentic Workflows with a Physical-Plan Compiler and Adaptive Runtime
  2. 2026From Static Templates to Dynamic Runtime Graphs: A Survey of Workflow Optimization for LLM Agents
  3. 2026Complete Cyclic Subtask Graphs for Tool-Using LLM Agents: Flexibility, Cost, and Bottlenecks in Long-Horizon Workflows
  4. 2026Pythia: Exploiting Workflow Predictability for Efficient Agent-Native LLM Serving
  5. 2026From Agent Loops to Structured Graphs:A Scheduler-Theoretic Framework for LLM Agent Execution
  6. 2026GraphBit: A Graph-based Agentic Framework for Non-Linear Agent Orchestration
  7. 2026HexAGenT: Efficient Agentic LLM Serving via Workflow- and Heterogeneity-Aware Scheduling
  8. 2026Autonomous Topology Mutation: Safe Runtime Restructuring for Multi-Agent LLM Systems with Capability, State, and Shadow Invariants