mlsanityData to 7 Oct 2026

LLMServingSim2.0: A Unified Simulator for Heterogeneous Hardware and Serving Techniques in LLM Infrastructure

arXiv 2511.07229 ↗First posted 10 Nov 2025cs.DC cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

This paper introduces LLMServingSim2.0, a system simulator designed for exploring heterogeneous hardware in large-scale LLM serving systems. LLMServingSim2.0 addresses two key limitations of its predecessor: (1) integrating hardware models into system-level simulators is non-trivial due to the lack of a clear abstraction, and (2) existing simulators support only a narrow subset of serving techniques, leaving no infrastructure that captures the breadth of approaches in modern LLM serving. To overcome these issues, LLMServingSim2.0 adopts trace-driven performance modeling, accompanied by an operator-level latency profiler, enabling the integration of new accelerators with a single command. It further embeds up-to-date serving techniques while exposing flexible interfaces for request routing, cache management, and scheduling policies. In a TPU case study, our profiler requires 18.5x fewer LoC and outperforms the predecessor's hardware-simulator integration, demonstrating LLMServingSim2.0's low-effort hardware extensibility. Our experiments further show that LLMServingSim2.0 reproduces GPU-based LLM serving with 1.9% error, while maintaining practical simulation time, making it a comprehensive platform for both hardware developers and LLM service providers.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 11 of its 13 references that are in the map. First posted on arXiv 10 Nov 2025.

Builds on

11 of 13 references.

References · 11

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  2. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  3. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 595
  4. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 536
  5. 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusioncited by 211
  6. 2024Vidur: A Large-Scale Simulation Framework For LLM Inferencecited by 115
  7. 2023Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inferencecited by 101
  8. 2024Duplex: A Device for Large Language Models with Mixture of Experts, Grouped Query Attention, and Continuous Batchingcited by 33
  9. 2024LLMServingSim: A HW/SW Co-Simulation Infrastructure for LLM Inference Serving at Scalecited by 30
  10. 2024APEX: An Extensible and Dynamism-Aware Simulator for Automated Parallel Execution in LLM Servingcited by 15
  11. 2025TokenSim: Enabling Hardware and Software Exploration for Large Language Model Inference Systemscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026Revati: Transparent GPU-Free Time-Warp Emulation for LLM Servingcited by 5
  2. 2026KernelSight-LM: A Kernel-Level LLM Inference Simulatorcited by 2
  3. 2026LLM-Emu: Native Runtime Emulation of LLM Inference via Profile-Driven Samplingcited by 0
  4. 2026Dooly: Configuration-Agnostic, Redundancy-Aware Profiling for LLM Inference Simulationcited by 0
  5. 2026LLMET: Enabling Cross-Layer Evaluation of Emerging M3D Memories for Energy-Efficient LLM Servingcited by 0
  6. 2026InFactPlanner: Planning Sustainable Geo-Distributed LLM Data Centerscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026LLMServingSim 2.0: A Unified Simulator for Heterogeneous and Disaggregated LLM Serving Infrastructure
  2. 2026NELSSA: A GPU-PNM Heterogeneous System for Mixed-Length LLM Serving via Length-based Request Placement
  3. 2025Agent.xpu: Efficient Scheduling of Agentic LLM Workloads on Heterogeneous SoC
  4. 2025DynaServe: Unified and Elastic Execution for Dynamic Disaggregated LLM Serving
  5. 2026SiliconBench: Speed, Memory, and Fidelity for LLM Serving on Unified-Memory Desktops
  6. 2025Sandwich: Joint Configuration Search and Hot-Switching for Efficient CPU LLM Serving
  7. 2026DualScale: Energy-Efficient Disaggregated LLM Serving via Phase-Aware Placement and DVFS
  8. 2026RouterWise: Joint Resource Allocation and Routing for Latency-Aware Multi-Model LLM Serving