mlsanityData to 7 Oct 2026

Reasoning Language Model Inference Serving Unveiled: An Empirical Study

arXiv 2510.18672 ↗First posted 21 Oct 2025cs.LG cs.AIcited by 1

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 1 more
Abstract (arXiv)

The reasoning large language model (RLLM) has been proven competitive in solving complex reasoning tasks such as mathematics, coding, compared to general LLM. However, the serving performance and behavior of RLLM remains unexplored, which may undermine the deployment and utilization of RLLM in real-world scenario. To close this gap, in this paper, we conduct a comprehensive study of RLLM service. We first perform a pilot study on comparing the serving performance between RLLM and traditional LLM and reveal that there are several distinct differences regarding serving behavior: (1) significant memory usage and fluctuations; (2) straggler requests; (3) adaptive running time; (4) domain preference. Then we further investigate whether existing inference optimization techniques are valid for RLLM. Our main takeaways are that model quantization methods and speculative decoding can improve service system efficiency with small compromise to RLLM accuracy, while prefix caching, KV cache quantization may even degrade accuracy or serving performance for small RLLM. Lastly, we conduct evaluation under real world workload modeled by Gamma distribution to verify our findings. Empirical results of real world workload evaluation across different dataset are aligned with our main findings regarding RLLM serving. We hope our work can provide the research community and industry with insights to advance RLLM inference serving.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 29 of its 88 references that are in the map. First posted on arXiv 21 Oct 2025.

Builds on

29 of 88 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Semantic Integrity Matters: Benchmarking and Preserving High-Density Reasoning in KV Cache Compression4 Feb

Other references · 28

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2020Scaling Laws for Neural Language Modelscited by 6,411
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  5. 2024Qwen2.5 Technical Reportcited by 4,713
  6. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  7. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  8. 2024OpenAI o1 System Cardcited by 1,220
  9. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  10. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  11. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 975
  12. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 700
16 more
  1. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  2. 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Testcited by 318
  3. 2025FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Servingcited by 248
  4. 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusioncited by 211
  5. 2025Seed1.5-Thinking: Advancing Superb Reasoning Models with Reinforcement Learningcited by 159
  6. 2025Phi-4-reasoning Technical Reportcited by 146
  7. 2023Fast Distributed Inference Serving for Large Language Modelscited by 143
  8. 2024Inference without Interference: Disaggregate LLM Inference for Mixed Downstream Workloadscited by 120
  9. 2024OpenRLHF: An Easy-to-use, Scalable and High-performance RLHF Frameworkcited by 70
  10. 2024MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Poolcited by 67
  11. 2024LLM Inference Serving: Survey of Recent Advances and Opportunitiescited by 46
  12. 2024A Survey of Low-bit Large Language Models: Basics, Systems, and Algorithmscited by 30
  13. 2025Taming the Titans: A Survey of Efficient LLM Inference Servingcited by 16
  14. 2024Do Large Language Models Need a Content Delivery Network?cited by 15
  15. 2024Etalon: Holistic Performance Evaluation Framework for LLM Inference Systemscited by 9
  16. 2024Accelerating Production LLMs with Combined Token/Embedding Speculatorscited by 8

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025DiffAdapt: Difficulty-Adaptive Reasoning for Token-Efficient LLM Inferencecited by 11

Similar, not cited

Close in wording, no citation either way

  1. 2025Comparative Analysis of Large Language Model Inference Serving Systems: A Performance Study of vLLM and HuggingFace TGI
  2. 2024Inference Scaling vs Reasoning: An Empirical Analysis of Compute-Optimal LLM Problem-Solving
  3. 2025Understanding and Mitigating Numerical Sources of Nondeterminism in LLM Inference
  4. 2024LLM Inference Unveiled: Survey and Roofline Model Insights
  5. 2025Hybrid Offline-online Scheduling Method for Large Language Model Inference Optimization
  6. 2026From Inference Engine to Inference Control Plane: Connecting vLLM, llm-d, and the Evolution of Efficient Distributed LLM Serving
  7. 2024ScaleLLM: A Resource-Frugal LLM Serving Framework by Optimizing End-to-End Efficiency
  8. 2024GUIDE: A Global Unified Inference Engine for Deploying Large Language Models in Heterogeneous Environments