mlsanityData to 14 Oct 2026

DWDP: Distributed Weight Data Parallelism for High-Performance LLM Inference on NVL72

arXiv 2604.01621 ↗First posted 2 Apr 2026cs.DC cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Large language model (LLM) inference increasingly depends on multi-GPU execution, yet existing inference parallelization strategies require layer-wise inter-rank synchronization, making end-to-end performance sensitive to workload imbalance. We present DWDP (Distributed Weight Data Parallelism), an inference parallelization strategy that preserves data-parallel execution while offloading MoE weights across peer GPUs and fetching missing experts on demand. By removing collective inter-rank synchronization, DWDP allows each GPU to progress independently. We further address the practical overheads of this design with two optimizations for split-weight management and asynchronous remote-weight prefetch. Implemented in TensorRT-LLM and evaluated with DeepSeek-R1 on GB200 NVL72, DWDP improves end-to-end output TPS/GPU by 8.8% at comparable TPS/user in the 20-100 TPS/user serving range under 8K input sequence length and 1K output sequence length.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 12 of its 26 references that are in the map. First posted on arXiv 2 Apr 2026.

Builds on

12 of 26 references.

References · 12

  1. 2021Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LMcited by 632
  2. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 617
  3. 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 587
  4. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 558
  5. 2022DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scalecited by 404
  6. 2025Seedance 1.0: Exploring the Boundaries of Video Generation Modelscited by 259
  7. 2022Tutel: Adaptive Mixture-of-Experts at Scalecited by 201
  8. 2024Preble: Efficient Distributed Prompt Scheduling for LLM Servingcited by 70
  9. 2025Barbarians at the Gate: How AI is Upending Systems Researchcited by 54
  10. 2025Strata: Hierarchical Context Caching for Long Context Language Model Servingcited by 26
  11. 2024When a language model is optimized for reasoning, does it still show embers of autoregression? An analysis of OpenAI o1cited by 22
  12. 2026Scalable Training of Mixture-of-Experts Models with Megatron Corecited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoningcited by 42

Similar, not cited

Close in wording, no citation either way

  1. 2026Characterizing Parallelism Strategies in LLM Inference: Fundamental Compute-Communication Trade-offs
  2. 2025Shift Parallelism: Low-Latency, High-Throughput LLM Inference for Dynamic Workloads
  3. 2026Scaling LLM Inference Beyond Amdahl`s Limits via Eliminating Non-Scalable Overheads ↗
  4. 2025Characterizing Communication Patterns in Distributed Large Language Model Inference
  5. 2025Striking the Right Balance between Compute and Copy: Improving LLM Inferencing Under Speculative Decoding
  6. 2025Parallax: Efficient LLM Inference Service over Decentralized Environment
  7. 2026Parallel Track Transformers: Enabling Fast GPU Inference with Reduced Synchronization
  8. 2024LLMEasyQuant: Scalable Quantization for Parallel and Distributed LLM Inference