# UBEP: Re-architecting Expert Parallelism Communication Library for Production Superpods

arXiv: https://arxiv.org/abs/2607.06202 · HTML: https://mlsanity.com/paper/2607.06202/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Efficient MoE training and inference systems, one of 12 directions in Mixture-of-experts.
- Builds on 18 of its 51 references that are in the map.
- First posted on arXiv 7 Jul 2026.

## Placed in

- Mixture-of-experts / [Efficient MoE training and inference systems](https://mlsanity.com/moe/efficient-moe-training-and-inference-systems/)

## Builds on (most cited first, 18 of 18)

- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [GLaM: Efficient Scaling of Language Models with Mixture-of-Experts](https://mlsanity.com/paper/2112.06905/) (2021)
- [DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale](https://mlsanity.com/paper/2201.05596/) (2022)
- [Tutel: Adaptive Mixture-of-Experts at Scale](https://mlsanity.com/paper/2206.03382/) (2022)
- [Accelerating Distributed MoE Training and Inference with Lina](https://mlsanity.com/paper/2210.17223/) (2022)
- [Comet: Fine-grained Computation-communication Overlapping for Mixture-of-Experts](https://mlsanity.com/paper/2502.19811/) (2025)
- [Insights into DeepSeek-V3: Scaling Challenges and Reflections on Hardware for AI Architectures](https://mlsanity.com/paper/2505.09343/) (2025)
- [Lancet: Accelerating Mixture-of-Experts Training via Whole Graph Computation-Communication Overlapping](https://mlsanity.com/paper/2404.19429/) (2024)
- [Optimizing Distributed ML Communication with Fused Computation-Collective Operations](https://mlsanity.com/paper/2305.06942/) (2023)
- [GRACE-MoE: Grouping and Replication with Locality-Aware Routing for Efficient Distributed MoE Inference](https://mlsanity.com/paper/2509.25041/) (2025)
- [MSCCL++: Rethinking GPU Communication Abstractions for AI Inference](https://mlsanity.com/paper/2504.09014/) (2025)
- [Accelerating Edge Inference for Distributed MoE Models with Latency-Optimized Expert Placement](https://mlsanity.com/paper/2508.12851/) (2025)
- [UB-Mesh: a Hierarchically Localized nD-FullMesh Datacenter Network Architecture](https://mlsanity.com/paper/2503.20377/) (2025)
- [LSH-MoE: Communication-efficient MoE Training via Locality-Sensitive Hashing](https://mlsanity.com/paper/2411.08446/) (2024)
- [HierMoE: Accelerating MoE Training with Hierarchical Token Deduplication and Expert Swap](https://mlsanity.com/paper/2508.09591/) (2025)
- [FUSCO: High-Performance Distributed Data Shuffling via Transformation-Communication Fusion](https://mlsanity.com/paper/2512.22036/) (2025)

## Abstract (arXiv)

The deployment of Mixture-of-Experts (MoE) models on production high-bandwidth superpods, such as NVIDIA's NVL72/576 and Huawei's CloudMatrix384, introduces critical challenges beyond raw interconnect bandwidth. While these systems provide unified global address spaces and high-bandwidth fabrics, their full potential for sparse MoE communication is hindered by three fundamental bottlenecks: (1) Strict execution serialization imposed by coarse-grained Bulk Synchronous Parallel (BSP) orchestration of interdependent communication phases; (2) Prohibitive synchronization overhead that fails to scale alongside high interconnect bandwidth; and (3) Severe load imbalance resulting from distance-agnostic scheduling of irregular token traffic. To eliminate these bottlenecks, we introduce UBEP (Unified-Bus Expert Parallelism), a production-ready communication library that rethinks MoE's All-to-All primitives for modern superpod architectures. Through large scale experiments, UBEP reduces All-to-All latency by up to 52.4% and MoE inference Time Per Output Token (TPOT) by up to 11.1%.