mlsanityData to 7 Oct 2026

TraceLab: Characterizing Coding Agent Workloads for LLM Serving

arXiv 2606.30560 ↗First posted 29 Jun 2026cs.LG cs.AI cs.PFcited by 14

On the map: LLM agents and tool use → Software engineering coding agents

also in 1 more
Abstract (arXiv)

Coding agents are rapidly becoming a major application of agentic LLMs, but serving them efficiently remains challenging. Progress on this challenge requires understanding real workload patterns, yet the data needed for such analysis is largely absent. Existing public traces and benchmarks do not capture real, day-to-day coding-agent usage across multiple agents and model families for serving-system analysis. To help fill this gap, we collect and release a trace of roughly 4,300 coding-agent sessions, containing about 350,000 LLM steps and 430,000 tool calls from our own day-to-day use of Claude Code and Codex. Our analysis shows that coding-agent workloads feature long autonomous loops, long contexts with short outputs, diverse and heavily-tailed tool calls, and high but imperfect prefix cache hit rates. These findings point to concrete opportunities for optimizing serving, including lower-overhead tool calling, append-length-aware prefill, semantic-aware tool-latency prediction, and improved KV-cache management around human-paced gaps. We release the dataset, trace collection pipeline, and analysis code at https://github.com/uw-syfi/TraceLab.git the project website is https://tracelab.cs.washington.edu.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 37 of its 77 references that are in the map. First posted on arXiv 29 Jun 2026.

Builds on

37 of 77 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025DeepCode: Open Agentic Coding8 Dec
  2. 2025Speed at the Cost of Quality: How Cursor AI Increases Short-Term Velocity and Long-Term Complexity in Open-Source Projects6 Nov
  3. 2025Continuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Live4 Nov

Other references · 34

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  3. 2021Program Synthesis with Large Language Modelscited by 3,509
  4. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  5. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  6. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  7. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
  8. 2021Measuring Coding Challenge Competence With APPScited by 879
  9. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 604
  10. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 571
  11. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 567
  12. 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 558
22 more
  1. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 527
  2. 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 515
  3. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 487
  4. 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 449
  5. 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 442
  6. 2024MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineeringcited by 429
  7. 2023Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Timecited by 428
  8. 2024MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attentioncited by 375
  9. 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 255
  10. 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusioncited by 211
  11. 2023CrossCodeEval: A Diverse and Multilingual Benchmark for Cross-File Code Completioncited by 181
  12. 2024Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttentioncited by 159
  13. 2023Fast Distributed Inference Serving for Large Language Modelscited by 143
  14. 2024Inference without Interference: Disaggregate LLM Inference for Mixed Downstream Workloadscited by 120
  15. 2024Parrot: Efficient Serving of LLM-based Applications with Semantic Variablecited by 96
  16. 2024ChunkAttention: Efficient Self-Attention with Prefix-Aware KV Cache and Two-Phase Partitioncited by 81
  17. 2024Hydragen: High-Throughput LLM Inference with Shared Prefixescited by 71
  18. 2024Preble: Efficient Distributed Prompt Scheduling for LLM Servingcited by 68
  19. 2024EPIC: Efficient Position-Independent Caching for Serving Large Language Modelscited by 45
  20. 2024Prompting Large Language Models to Tackle the Full Software Development Lifecycle: A Case Studycited by 40
  21. 2026DualPath: Breaking the Storage Bandwidth Bottleneck in Agentic LLM Inferencecited by 15
  22. 2026Programming by Chat: A Large-Scale Behavioral Analysis of 11,579 Real-World AI-Assisted IDE Sessionscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 14.

14 papers.

Sort:

By area:

  1. 2026Agentic Coding in the Wild: Characterizing GitHub Copilot Traces at Production Scalecited by 9
  2. 2026TokTier: Exact Stateful CPU+GPU Tokenization for Agentic LLM Servingcited by 2
  3. 2026Scheduling Mixed RL Rollouts Beyond Prefix Localitycited by 1
  4. 2026FreeToken: Efficient Edge-Native MoE Serving with Bandwidth-Adaptive Executioncited by 1
  5. 2026Control the Harness, Control the Cost: Routing and Governing AI Coding Agents in the Enterprisecited by 1
  6. 2026REVISE: Validity-Guided Recovery for Online Revisions in Agent Workflowscited by 0
  7. 2026Ask the Tool, Don't Guess: Agent Tool Calls Hold Their Progress, and the Serving System Should Read Itcited by 0
  8. 2026Crossflow: Prefill-Decode Elasticity for Agentic LLM Servingcited by 0
  9. 2026When Fancy Eviction Fails: Rethinking Cache Replacement For LLM Prefix Reusecited by 0
  10. 2026DynBranch: Speculative Subgraph Reuse for Dynamic Agentic LLM Servingcited by 0
  11. 2026KVCMAS: Efficient KV cache Correction for Shared Context in Multi-Agent Systemscited by 0
  12. 2026TokenCast: Forecasting Token Consumption During LLM Agent Executioncited by 0
2 more
  1. 2026Persistent Context Graphs for Efficient Memory Compaction in LLM Agentscited by 0
  2. 2026Code Understanding is a Bottleneck for Coding Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026CacheWise: Understanding Workloads and Optimizing KVCache Management for Efficiently Serving LLM Coding Agents
  2. 2026Inside the Scaffold: A Source-Code Taxonomy of Coding Agent Architectures
  3. 2026Automata from Agent Traces: Failure and Next-Step Prediction
  4. 2026FormulaCode: Evaluating Agentic Optimization on Large Codebases
  5. 2026Insights Generator: Systematic Corpus-Level Trace Diagnostics for LLM Agents
  6. 2026ProcCtrlBench: Evaluating Process-Level Defects and Control Preservation in LLM Coding Agents
  7. 2026AgentPerfBench: A Benchmarking and Evaluation Suite for Inference Performance of Agentic LLMs
  8. 2026A Phased Workflow for Operating LLM-Based Coding Agents