mlsanityData to 13 Oct 2026

Simthesizer: An Agent-Driven Simulation Framework for LLM Serving Systems

arXiv 2608.24650 ↗First posted 25 Aug 2026cs.AR cs.AIcited by 0

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

System-level simulation is an essential tool for exploring the rapidly expanding design space of LLM serving systems, where real deployments remain costly and often infeasible. However, modern LLM serving now evolves faster than human-driven simulator development can track, and emerging workloads and mechanisms, from agentic workflows to disaggregated serving, no longer fit the monolithic simulation pipeline that existing simulators assume. Each new mechanism therefore demands an invasive rewrite, leaving a widening development gap between deployed serving systems and the simulators that model them. To close this gap, we present Simthesizer, a framework that realizes agent-driven simulator development. Simthesizer introduces a composable simulator infrastructure that uniformly expresses the complete serving workflow, including the control decisions that coordinate it, and realizes it as a unified dynamic graph in Simthesizer simulator. Synthesizer agent, a harnessed coding agent, then lowers natural-language feature requests onto this abstraction under simulator-specific guardrails and fidelity validation, evolving one shared simulator instead of building a new one for every feature. Under the same coding agent and harnesses, extensions built on Simthesizer follow a vLLM-based real system with 2.51% average throughput error, versus 6.03% for extensions built on existing simulators. On identical workloads, Simthesizer also simulates up to 284.96x and 23.19x faster than two state-of-the-art simulators, LLMServingSim2.0 and Vidur, respectively.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 44 of its 60 references that are in the map. First posted on arXiv 25 Aug 2026.

Builds on

44 of 60 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Pie: A Programmable Serving System for Emerging LLM Applications28 Oct
  2. 2025CudaForge: An Agent Framework with Hardware Feedback for CUDA Kernel Optimization23 Oct
  3. 2025AIOpsLab: A Holistic Framework to Evaluate AI Agents for Enabling Autonomous Clouds12 Jan
  4. 2024SuffixDecoding: Extreme Speculative Decoding for Emerging AI Applications7 Nov
  5. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation16 Aug

Other references · 39

  1. 2025Qwen3 Technical Reportcited by 8,510
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,502
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,831
  4. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,156
  5. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,117
  6. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,274
  7. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,777
  8. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,625
  9. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,335
  10. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,180
  11. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 602
  12. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertaintycited by 569
27 more
  1. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 545
  2. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Servecited by 514
  3. 2024EAGLE-2: Faster Inference of Language Models with Dynamic Draft Treescited by 362
  4. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verificationcited by 361
  5. 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Testcited by 319
  6. 2024Llumnix: Dynamic Scheduling for Large Language Model Servingcited by 169
  7. 2024Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttentioncited by 160
  8. 2024Vidur: A Large-Scale Simulation Framework For LLM Inferencecited by 116
  9. 2024LoongServe: Efficiently Serving Long-Context Large Language Models with Elastic Sequence Parallelismcited by 110
  10. 2024Parrot: Efficient Serving of LLM-based Applications with Semantic Variablecited by 98
  11. 2024vAttention: Dynamic Memory Management for Serving LLMs without PagedAttentioncited by 96
  12. 2024NanoFlow: Towards Optimal Large Language Model Serving Throughputcited by 88
  13. 2023Stateful Large Language Model Serving with Pensievecited by 60
  14. 2024POD-Attention: Unlocking Full Prefill-Decode Overlap for Faster LLM Inferencecited by 46
  15. 2025KVCache Cache in the Wild: Characterizing and Optimizing KVCache Cache at a Large Cloud Providercited by 41
  16. 2024LLMServingSim: A HW/SW Co-Simulation Infrastructure for LLM Inference Serving at Scalecited by 30
  17. 2024LLM-Vectorizer: LLM-based Verified Loop Vectorizercited by 25
  18. 2025KNighter: Transforming Static Analysis with LLM-Synthesized Checkerscited by 23
  19. 2023vTrain: A Simulation Framework for Evaluating Cost-effective and Compute-optimal Large Language Model Trainingcited by 22
  20. 2023DeltaZip: Efficient Serving of Multiple Full-Model-Tuned LLMscited by 17
  21. 2024APEX: An Extensible and Dynamism-Aware Simulator for Automated Parallel Execution in LLM Servingcited by 15
  22. 2026LLMServingSim 2.0: A Unified Simulator for Heterogeneous and Disaggregated LLM Serving Infrastructurecited by 8
  23. 2025Pimba: A Processing-in-Memory Acceleration for Post-Transformer Large Language Model Servingcited by 7
  24. 2024ONNXim: A Fast, Cycle-level Multi-core NPU Simulatorcited by 5
  25. 2025ADOR: A Design Exploration Framework for LLM Serving with Enhanced Latency and Throughputcited by 4
  26. 2025LOOPRAG: Enhancing Loop Transformation Optimization with Retrieval-Augmented Large Language Modelscited by 1
  27. 2026PASCAL: A Phase-Aware Scheduling Algorithm for Serving Reasoning-based Large Language Modelscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026AgentServeSim: Serving-System Simulation and Policy Search for LLM Agent Programs
  2. 2026VibeServe: Can AI Agents Build Bespoke LLM Serving Systems?
  3. 2026A Policy-Driven Runtime Layer for Agentic LLM Serving
  4. 2026Frontier: Towards Comprehensive and Accurate LLM Inference Simulation
  5. 2026Pythia: Exploiting Workflow Predictability for Efficient Agent-Native LLM Serving
  6. 2026SimuAgent: An LLM-Based Simulink Modeling Assistant Enhanced with Reinforcement Learning
  7. 2026Benchmarking LLM Serving Systems for Agentic AI Workloads with XPerf
  8. 2026Chimera: Latency- and Performance-Aware Multi-agent Serving for Heterogeneous LLMs