# Incast-Free MoE Rate-Based Scheduling

arXiv: https://arxiv.org/abs/2607.26340 · HTML: https://mlsanity.com/paper/2607.26340/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Efficient MoE training and inference systems, one of 12 directions in Mixture-of-experts.
- Builds on 7 of its 29 references that are in the map.
- First posted on arXiv 28 Jul 2026.

## Placed in

- Mixture-of-experts / [Efficient MoE training and inference systems](https://mlsanity.com/moe/efficient-moe-training-and-inference-systems/)

## Builds on (most cited first, 7 of 7)

- [Demystifying the Nvidia Ampere Architecture through Microbenchmarking and Instruction-level Analysis](https://mlsanity.com/paper/2208.11174/) (2022)
- [InfiniteHBD: Building Datacenter-Scale High-Bandwidth Domain for LLM with Optical Circuit Switching Transceivers](https://mlsanity.com/paper/2502.03885/) (2025)
- [REPS: Recycled Entropy Packet Spraying for Adaptive Load Balancing and Failure Mitigation](https://mlsanity.com/paper/2407.21625/) (2024)
- [BigMac: A Communication-Efficient Mixture-of-Experts Model Structure for Fast Training and Inference](https://mlsanity.com/paper/2502.16927/) (2025)
- [STrack: A Reliable Multipath Transport for AI/ML Clusters](https://mlsanity.com/paper/2407.15266/) (2024)
- [FAST: An Efficient Scheduler for All-to-All GPU Communication](https://mlsanity.com/paper/2505.09764/) (2025)
- [Dynamic Hierarchical Birkhoff-von Neumann Decomposition for All-to-All GPU Communication](https://mlsanity.com/paper/2602.22756/) (2026)

## Abstract (arXiv)

Mixture of Experts (MoE) architectures have become key to large language models; however, their typical round-robin (RR) scheduling introduces significant bottlenecks. In this paper, we demonstrate that RR causes a previously-undiscovered exponential incast phenomenon with MoE traffic. We propose an alternative proactive fair scheduling framework tailored for MoE workloads, which effectively prevents fabric oversubscription. We also outline how it can be implemented in NICs. Finally, through extensive simulations with real and synthetic workloads, we demonstrate that this framework consistently eliminates incast, maintains a near-100% link utilization, and reduces Collective Completion Time (CCT).