mlsanityData to 7 Oct 2026

TokenSim: Enabling Hardware and Software Exploration for Large Language Model Inference Systems

arXiv 2503.08415 ↗First posted 11 Mar 2025cs.DCcited by 6

On the map: Efficient LLM inference and serving → Serving systems and request scheduling

Abstract (arXiv)

The increasing demand for large language model (LLM) serving has necessitated significant advancements in the optimization and profiling of LLM inference systems. As these models become integral to a wide range of applications, the need for efficient and scalable serving solutions has grown exponentially. This work introduces TokenSim, a comprehensive hardware and software exploration system designed specifically for LLM inference. TokenSim is characterized by its support for extensible system optimizations including scheduling and memory management. We validate the results with systems running with realworld datasets, achieving an error rate of less than 1%. Furthermore, TokenSim facilitates various insightful explorations into the performance and optimization of LLM serving systems.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 15 of its 22 references that are in the map. First posted on arXiv 11 Mar 2025.

Builds on

15 of 22 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024LLMServingSim: A HW/SW Co-Simulation Infrastructure for LLM Inference Serving at Scale10 Aug
  2. 2024MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Pool25 Jun
  3. 2024Demystifying AI Platform Design for Distributed Inference of Next-Generation LLM models3 Jun
  4. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving18 Jan

Other references · 11

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  7. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 536
  8. 2022Efficiently Scaling Transformer Inferencecited by 529
  9. 2024Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttentioncited by 159
  10. 2024Vidur: A Large-Scale Simulation Framework For LLM Inferencecited by 115
  11. 2023A Hardware Evaluation Framework for Large Language Model Inferencecited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 5.

6 papers.

  1. 2026LLMServingSim 2.0: A Unified Simulator for Heterogeneous and Disaggregated LLM Serving Infrastructurecited by 8
  2. 2025LLMServingSim2.0: A Unified Simulator for Heterogeneous Hardware and Serving Techniques in LLM Infrastructurecited by 6
  3. 2026Revati: Transparent GPU-Free Time-Warp Emulation for LLM Servingcited by 5
  4. 2026Characterization of Multi-Model Agentic AI Systems on General Tasks via Trace-Driven Simulationcited by 3
  5. 2026KernelSight-LM: A Kernel-Level LLM Inference Simulatorcited by 2
  6. 2026Hydra: Phase-Aware Workload Characterization of LLM Inference across Edge SoC Generations, Backends, and Quantization Levelscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2023Efficient LLM Inference on CPUs
  2. 2024LLM Inference Unveiled: Survey and Roofline Model Insights
  3. 2024GUIDE: A Global Unified Inference Engine for Deploying Large Language Models in Heterogeneous Environments
  4. 2024Large Language Model Inference Acceleration: A Comprehensive Hardware Perspective
  5. 2025AccLLM: Accelerating Long-Context LLM Inference Via Algorithm-Hardware Co-Design
  6. 2023Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline
  7. 2024A Survey on Large Language Model Acceleration based on KV Cache Management
  8. 2025ReaLM: Reliable and Efficient Large Language Model Inference with Statistical Algorithm-Based Fault Tolerance