mlsanityData to 14 Oct 2026

FAST: An Efficient Scheduler for All-to-All GPU Communication

arXiv 2505.09764 ↗First posted 14 May 2025cs.DC cs.NIcited by 3

On the map: Mixture-of-experts → Efficient MoE training and inference systems

Abstract (arXiv)

All-to-All(v) communication is a critical primitive in modern machine learning workloads, particularly mixture-of-experts (MoE) models. Unfortunately, efficient scheduling is challenging due to workload skew, heterogeneous two-tier fabrics, and incast congestion, compounded by the dynamic nature of MoE workloads, where traffic shifts every few hundred milliseconds. Existing schedulers are hardly scalable, incurring seconds to hours of synthesis time, making them impractical. We present FAST, an efficient All-to-All(v) scheduler. FAST addresses skew through intra-server rebalancing and enforces balanced, one-to-one scale-out transfers that avoid incast. Evaluated extensively on both NVIDIA H200 and AMD MI300X clusters, FAST consistently outperforms state-of-the-art solutions on skewed workloads while reducing synthesis time by orders of magnitude.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Efficient MoE training and inference systems, one of 12 directions in Mixture-of-experts. First posted on arXiv 14 May 2025.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Rethinking LLM Inference Bottlenecks: Insights from Latent Attention and Mixture-of-Expertscited by 4
  2. 2026Incast-Free MoE Rate-Based Schedulingcited by 0
  3. 2026Efficient Expert-Parallel Communication on PCIe-Connected Consumer GPUscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Weave: Fine-Grained Dynamic SM Scheduling in an MoE Megakernel for Compute-Communication Overlap
  2. 2024Optimizing Mixture-of-Experts Inference Time Combining Model Deployment and Communication Scheduling
  3. 2026Accelerating MoE with Dynamic In-Switch Computing on Multi-GPUs
  4. 2026Automated Tensor Scheduling for Hybrid CPU-GPU LLM Inference on Consumer Devices
  5. 2023Optimizing Distributed ML Communication with Fused Computation-Collective Operations
  6. 2026Fine-grained Computation-Communication Overlap via Tile-level Signaling and Scheduling for Mixture-of-Experts
  7. 2025Toward Cost-Efficient Serving of Mixture-of-Experts with Asynchrony
  8. 2025MoETuner: Optimized Mixture of Expert Serving with Balanced Expert Placement and Token Routing