mlsanityData to 14 Oct 2026

Frontier: Towards Comprehensive and Accurate LLM Inference Simulation

arXiv 2605.21312 ↗First posted 20 May 2026cs.DC cs.AI cs.LGcited by 4

On the map: Efficient LLM inference and serving → Serving systems and request scheduling

Abstract (arXiv)

Modern LLM serving is no longer homogeneous or monolithic. Production systems now combine disaggregated execution, complex parallelism, runtime optimizations, and stateful workloads such as reasoning, agents, and RL rollouts. Simulation is attractive for exploring this growing design space, yet existing simulators lack the architectural completeness and decision-grade fidelity it demands. Their monolithic-replica abstractions are ill-suited to disaggregated serving, while average-case analytical proxies can distort SLA predictions and even reverse optimization conclusions. We present Frontier, a discrete-event simulator for modern LLM inference serving. Frontier features a disaggregated abstraction. It captures the structure and dynamics of modern serving systems by modeling co-location, Prefill-Decode Disaggregation (PDD), and Attention-FFN Disaggregation (AFD) with role-specific cluster workers, incorporating key runtime optimizations (e.g., CUDA Graphs, speculative decoding) within the scheduler-batch-engine loop, and supporting stateful requests for emerging workloads. It further provides accurate and generalizable predictions of computation, communication, and memory costs across diverse serving scenarios with complex workload compositions. On 16-H800 GPU testbed, Frontier achieves an average throughput error below 4%. Compared with state-of-the-art simulators, it reduces end-to-end latency error from 44.9% to 6.4% under co-location and from 51.7% to 2.6% under disaggregation. It scales to over 1K GPUs on commodity CPUs and enables new use cases such as SLA-dependent Pareto frontier exploration, heterogeneous disaggregated allocation, agentic reasoning scheduling validation, and RL post-training reconfiguration. We release Frontier at https://github.com/NetX-lab/Frontier.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 25 of its 39 references that are in the map. First posted on arXiv 20 May 2026.

Builds on

25 of 39 references.

Closest ancestors · 5

Same direction, newest first

  1. 2026LLMServingSim 2.0: A Unified Simulator for Heterogeneous and Disaggregated LLM Serving Infrastructure26 Feb
  2. 2026AIConfigurator: Lightning-Fast Configuration Optimization for Multi-Framework LLM Serving9 Jan
  3. 2024APEX: An Extensible and Dynamism-Aware Simulator for Automated Parallel Execution in LLM Serving26 Nov
  4. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve4 Mar
  5. 2023SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills31 Aug

Other references · 20

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  4. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  5. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  6. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  7. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  8. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
  9. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  10. 2025FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Servingcited by 258
  11. 2024Vidur: A Large-Scale Simulation Framework For LLM Inferencecited by 117
  12. 2022Accelerating Distributed MoE Training and Inference with Linacited by 109
8 more
  1. 2023A Hybrid Tensor-Expert-Data Parallelism Approach to Optimize Mixture-of-Experts Trainingcited by 47
  2. 2023ASTRA-sim2.0: Modeling Hierarchical Networks and Disaggregated Systems for Large-model Training at Scalecited by 47
  3. 2025MegaScale-Infer: Serving Mixture-of-Experts at Scale with Disaggregated Expert Parallelismcited by 32
  4. 2023Proteus: Simulating the Performance of Distributed DNN Trainingcited by 10
  5. 2026Heddle: A Distributed Orchestration System for Agentic RL Rolloutcited by 8
  6. 2026Revati: Transparent GPU-Free Time-Warp Emulation for LLM Servingcited by 5
  7. 2026OrchestrRL: Dynamic Compute and Network Orchestration for Disaggregated RLcited by 5
  8. 2024Echo: Simulating Distributed Training At Scalecited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Libra: Efficient Resource Management for Agentic RL Post-Trainingcited by 4
  2. 2026AFD-Ledger: Deployment Provisioning for Attention--FFN Disaggregationcited by 0
  3. 2026RoofLang: Enabling AI-Driven Architecting of LLM Inference Systemscited by 0
  4. 2026Analytical Power-Aware Provisioning for Prefill-Decode Disaggregated AI Inferencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025MIST: A Co-Design Framework for Heterogeneous, Multi-Stage LLM Inference
  2. 2024LLMServingSim: A HW/SW Co-Simulation Infrastructure for LLM Inference Serving at Scale
  3. 2026From Inference Engine to Inference Control Plane: Connecting vLLM, llm-d, and the Evolution of Efficient Distributed LLM Serving
  4. 2026When Does Disaggregation Pay? Simulating Prefill--Decode--Attention--FFN Specialization for Agentic LLM Inference
  5. 2025Frontier: Simulating the Next Generation of LLM Inference Systems
  6. 2026SageSched: Efficient LLM Scheduling Confronting Demand Uncertainty and Hybridity
  7. 2026InferenceBench: A Benchmark for Open-Ended LLM Inference Optimization by AI Agents
  8. 2026Beyond Prediction: Tail-Aware Scheduling for LLM Inference