# UCCL-EP: Portable Expert-Parallel Communication

arXiv: https://arxiv.org/abs/2512.19849 · HTML: https://mlsanity.com/paper/2512.19849/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Placed in Efficient MoE training and inference systems, one of 12 directions in Mixture-of-experts.
- Builds on 18 of its 57 references that are in the map.
- First posted on arXiv 22 Dec 2025.

## Placed in

- Mixture-of-experts / [Efficient MoE training and inference systems](https://mlsanity.com/moe/efficient-moe-training-and-inference-systems/)

## Builds on (most cited first, 18 of 18)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Mixtral of Experts](https://mlsanity.com/paper/2401.04088/) (2024)
- [SGLang: Efficient Execution of Structured Language Model Programs](https://mlsanity.com/paper/2312.07104/) (2023)
- [GLaM: Efficient Scaling of Language Models with Mixture-of-Experts](https://mlsanity.com/paper/2112.06905/) (2021)
- [DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models](https://mlsanity.com/paper/2401.06066/) (2024)
- [DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models](https://mlsanity.com/paper/2512.02556/) (2025)
- [OLMoE: Open Mixture-of-Experts Language Models](https://mlsanity.com/paper/2409.02060/) (2024)
- [MegaBlocks: Efficient Sparse Training with Mixture-of-Experts](https://mlsanity.com/paper/2211.15841/) (2022)
- [Helix: Serving Large Language Models over Heterogeneous GPUs and Network via Max-Flow](https://mlsanity.com/paper/2406.01566/) (2024)
- [Characterization of Large Language Model Development in the Datacenter](https://mlsanity.com/paper/2403.07648/) (2024)
- [Insights into DeepSeek-V3: Scaling Challenges and Reflections on Hardware for AI Architectures](https://mlsanity.com/paper/2505.09343/) (2025)
- [M'elange: Cost Efficient Large Language Model Serving by Exploiting GPU Heterogeneity](https://mlsanity.com/paper/2404.14527/) (2024)
- [SkyServe: Serving AI Models across Regions and Clouds with Spot Instances](https://mlsanity.com/paper/2411.01438/) (2024)
- [HexGen-2: Disaggregated Generative Inference of LLMs in Heterogeneous Environment](https://mlsanity.com/paper/2502.07903/) (2025)
- [An Extensible Software Transport Layer for GPU Networking](https://mlsanity.com/paper/2504.17307/) (2025)
- [fabric-lib: RDMA Point-to-Point Communication for LLM Systems](https://mlsanity.com/paper/2510.27656/) (2025)
- [Taming the Chaos: Coordinated Autoscaling for Heterogeneous and Disaggregated LLM Inference](https://mlsanity.com/paper/2508.19559/) (2025)
- [Collective Communication for 100k+ GPUs](https://mlsanity.com/paper/2510.20171/) (2025)

## Built on by (most cited first, 12 of 12)

- [UltraEP: Unleash MoE Training and Inference on Rack-Scale Nodes with Near-Optimal Load Balancing](https://mlsanity.com/paper/2606.04101/) (2026)
- [NCCL EP: Towards a Unified Expert Parallel Communication API for NCCL](https://mlsanity.com/paper/2603.13606/) (2026)
- [Making Every Verified Token Count: Adaptive Verification for MoE Speculative Decoding](https://mlsanity.com/paper/2605.00342/) (2026)
- [Surviving Partial Rank Failures in Wide Expert-Parallel MoE Inference](https://mlsanity.com/paper/2605.10670/) (2026)
- [Move the Query, Not the Cache: Characterizing Cross-Instance Latent Attention Redistribution Across GPU Fabrics](https://mlsanity.com/paper/2606.01502/) (2026)
- [GF-DiT: Scheduling Parallelism for Diffusion Transformer Serving](https://mlsanity.com/paper/2606.13501/) (2026)
- [Moebius: Serving Mixture-of-Expert Models with Seamless Runtime Parallelism Switch](https://mlsanity.com/paper/2606.26607/) (2026)
- [Searching for Plans You Can Actually Build: A Realizability-Aware Full-Space Optimizer for MoE Training and Serving](https://mlsanity.com/paper/2607.18631/) (2026)
- [NanoCP: Request-Level Dynamic Context Parallelism for Data-Expert Parallel Decoding](https://mlsanity.com/paper/2605.21100/) (2026)
- [mKernel: Fast Multi-GPU, Multi-Node Fused Kernels](https://mlsanity.com/paper/2609.13585/) (2026)
- [Efficient Expert-Parallel Communication on PCIe-Connected Consumer GPUs](https://mlsanity.com/paper/2609.40093/) (2026)
- [Memory-Efficient Expert Routing for Distributed MoE Training](https://mlsanity.com/paper/2610.07333/) (2026)

## Abstract (arXiv)

Mixture-of-Experts (MoE) workloads rely on expert parallelism (EP) to achieve high GPU efficiency. State-of-the-art EP communication systems such as DeepEP demonstrate strong performance but exhibit poor portability across heterogeneous GPU and NIC platforms. The poor portability is rooted in architecture: GPU-initiated token-level RDMA communication requires tight vertical integration between GPUs and NICs, e.g., GPU writes to NIC driver/MMIO interfaces. We present UCCL-EP, a portable EP communication system that delivers DeepEP-level performance across heterogeneous GPU and NIC hardware. UCCL-EP replaces GPU-initiated RDMA with a high-throughput GPU-CPU control channel: compact token-routing commands are transferred to multithreaded CPU proxies, which then issue GPUDirect RDMA operations on behalf of GPUs. UCCL-EP further emulates various ordering semantics required by specialized EP communication modes using RDMA immediate data, enabling correctness on NICs that lack such ordering, e.g., AWS EFA. We implement UCCL-EP on NVIDIA and AMD GPUs with EFA and Broadcom NICs. On EFA, it outperforms the best existing EP solution by up to 2.1× for dispatch and combine throughput. On NVIDIA-only platform, UCCL-EP achieves comparable performance to the original DeepEP. UCCL-EP also improves token throughput on SGLang by up to 40% on the NVIDIA+EFA platform, and improves DeepSeek-V3 training throughput over the AMD Primus/Megatron-LM framework by up to 45% on a 16-node AMD+Broadcom platform.