mlsanityData to 14 Oct 2026

Charon: A Unified and Fine-Grained Simulator for Large-Scale LLM Training and Inference

arXiv 2605.17164 ↗First posted 16 May 2026cs.DC cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Deploying large-scale LLM training and inference with optimal performance is exceptionally challenging due to a complex design space of parallelism strategies, system optimizations, and hardware configurations. Accurate and rapid performance simulation is critical for guiding optimization efforts and system studies by validating "what-if" Hooker Figure hypotheses. To address this, we introduce Charon, a unified, modular, and fine-grained simulator for accurately predicting LLM performance. Experiments show Charon achieves high accuracy across different models and configurations, with an overall prediction error consistently under 5.35%, and even under 3.74% for training with a large-scale GPU cluster. In a practical inference deployment case, Charon discovered a configuration that improved system throughput over an engineering-tuned baseline, demonstrating its significant real-world value.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 13 of its 19 references that are in the map. First posted on arXiv 16 May 2026.

Builds on

13 of 19 references.

References · 13

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  3. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  4. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  5. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
  6. 2021Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LMcited by 632
  7. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 559
  8. 2021Sequence Parallelism: Long Sequence Training from System Perspectivecited by 126
  9. 2024Vidur: A Large-Scale Simulation Framework For LLM Inferencecited by 117
  10. 2023ASTRA-sim2.0: Modeling Hierarchical Networks and Disaggregated Systems for Large-model Training at Scalecited by 47
  11. 2024LLMServingSim: A HW/SW Co-Simulation Infrastructure for LLM Inference Serving at Scalecited by 31
  12. 2025VeOmni: Scaling Any Modality Model Training with Model-Centric Distributed Recipe Zoocited by 20
1 more
  1. 2025Simulating LLM training workloads for heterogeneous compute and network infrastructurecited by 1

Built on by

Not cited yet.