UBEP: Re-architecting Expert Parallelism Communication Library for Production Superpods
On the map: Mixture-of-experts → Efficient MoE training and inference systems
Abstract (arXiv)
The deployment of Mixture-of-Experts (MoE) models on production high-bandwidth superpods, such as NVIDIA's NVL72/576 and Huawei's CloudMatrix384, introduces critical challenges beyond raw interconnect bandwidth. While these systems provide unified global address spaces and high-bandwidth fabrics, their full potential for sparse MoE communication is hindered by three fundamental bottlenecks: (1) Strict execution serialization imposed by coarse-grained Bulk Synchronous Parallel (BSP) orchestration of interdependent communication phases; (2) Prohibitive synchronization overhead that fails to scale alongside high interconnect bandwidth; and (3) Severe load imbalance resulting from distance-agnostic scheduling of irregular token traffic. To eliminate these bottlenecks, we introduce UBEP (Unified-Bus Expert Parallelism), a production-ready communication library that rethinks MoE's All-to-All primitives for modern superpod architectures. Through large scale experiments, UBEP reduces All-to-All latency by up to 52.4% and MoE inference Time Per Output Token (TPOT) by up to 11.1%.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Efficient MoE training and inference systems, one of 12 directions in Mixture-of-experts. Builds on 18 of its 51 references that are in the map. First posted on arXiv 7 Jul 2026.
Builds on
18 of 51 references.
Closest ancestors · 11
Same direction, newest first
- 2025FUSCO: High-Performance Distributed Data Shuffling via Transformation-Communication Fusion26 Dec
- 2025GRACE-MoE: Grouping and Replication with Locality-Aware Routing for Efficient Distributed MoE Inference29 Sep
- 2025Accelerating Edge Inference for Distributed MoE Models with Latency-Optimized Expert Placement18 Aug
- 2025HierMoE: Accelerating MoE Training with Hierarchical Token Deduplication and Expert Swap13 Aug
- 2025Insights into DeepSeek-V3: Scaling Challenges and Reflections on Hardware for AI Architectures14 May
- 2025Comet: Fine-grained Computation-communication Overlapping for Mixture-of-Experts27 Feb
- 2024LSH-MoE: Communication-efficient MoE Training via Locality-Sensitive Hashing13 Nov
- 2024Lancet: Accelerating Mixture-of-Experts Training via Whole Graph Computation-Communication Overlapping30 Apr
- 2022Accelerating Distributed MoE Training and Inference with Lina31 Oct
- 2022Tutel: Adaptive Mixture-of-Experts at Scale7 Jun
- 2022DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale14 Jan
Other references · 7
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
- 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
- 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
- 2023Optimizing Distributed ML Communication with Fused Computation-Collective Operationscited by 15
- 2025MSCCL++: Rethinking GPU Communication Abstractions for AI Inferencecited by 9
- 2025UB-Mesh: a Hierarchically Localized nD-FullMesh Datacenter Network Architecturecited by 6
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026MoE-Hub: Taming Software Complexity for Seamless MoE Overlap with Hardware-Accelerated Communication on Multi-GPU Systems
- 2026Weave: Fine-Grained Dynamic SM Scheduling in an MoE Megakernel for Compute-Communication Overlap
- 2026UltraEP: Unleash MoE Training and Inference on Rack-Scale Nodes with Near-Optimal Load Balancing
- 2025UCCL-EP: Portable Expert-Parallel Communication
- 2026Accelerating MoE with Dynamic In-Switch Computing on Multi-GPUs
- 2026StrataCL: Fabric-Native Communication Library for Production Supernodes ↗
- 2026Efficient Expert-Parallel Communication on PCIe-Connected Consumer GPUs
- 2026Analytical Resource Management for Fine-grained MoE Computation-Communication Overlap