mlsanityData to 7 Oct 2026

TAOT: Topology-Aware Optimal Transport for Dynamic Expert Replica Placement in MoE Training

arXiv 2608.03676 ↗First posted 4 Aug 2026cs.DCcited by 1

On the map: Mixture-of-experts → Efficient MoE training and inference systems

Abstract (arXiv)

Mixture-of-Experts (MoE) has become a key architecture for scaling large language models (LLMs), yet its dynamic routing causes severe load imbalance in expert-parallel training. Existing dynamic-replica methods copy hot experts onto idle ranks to share computation, but they optimize load balance alone and ignore the cost of moving expert weights across a multi-node topology, so the resulting cross-node communication can outweigh the balancing gain and inflate training cost. We present TAOT, a topology-aware optimal transport method for dynamic expert-replica placement. TAOT models the overload on hot ranks and the spare capacity on lightly loaded ranks as a balanced entropy-regularized optimal transport problem with a communication-cost matrix, solves it with Sinkhorn-Knopp iterations to produce rank-level flow hints, and combines integer replica matching with token assignment into an executable schedule. At the system level, it overlaps guest-weight transfer with home-expert computation to hide the communication overhead. Experiments show TAOT achieves a 1.43x end-to-end MoE training speedup, reaches balance quality competitive with or better than existing state-of-the-art methods, and attains the lowest weighted expert-communication cost across all configurations, with up to a 74% reduction.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Efficient MoE training and inference systems, one of 12 directions in Mixture-of-experts. Builds on 18 of its 29 references that are in the map. First posted on arXiv 4 Aug 2026.

Builds on

18 of 29 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026FEPLB: Exploiting Copy Engines for Nearly Free MoE Load Balancing in Distributed Training21 Apr
  2. 2026LAER-MoE: Load-Adaptive Expert Re-layout for Efficient Mixture-of-Experts Training12 Feb
  3. 2026Least-Loaded Expert Parallelism: Load Balancing An Imbalanced Mixture-of-Experts23 Jan
  4. 2025SYMI: Efficient Mixture-of-Experts Training via Model and Optimizer State Decoupling28 Apr
  5. 2022Tutel: Adaptive Mixture-of-Experts at Scale7 Jun
  6. 2022DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale14 Jan

Other references · 12

  1. 2017Attention Is All You Needcited by 59,932
  2. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  3. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  4. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  5. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  6. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  7. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  8. 2022Mixture-of-Experts with Expert Choice Routingcited by 611
  9. 2022ST-MoE: Designing Stable and Transferable Sparse Expert Modelscited by 391
  10. 2021BASE Layers: Simplifying Training of Large, Sparse Modelscited by 315
  11. 2021Hash Layers For Large Sparse Modelscited by 236
  12. 2025FLEX-MoE: Federated Mixture-of-Experts with Load-balanced Expert Assignment for Edge Computingcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026TEMPO: Makespan-Aware Expert-Parallel Load Balancing Across Memory- and Compute-Bound Regimescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026FoMoE: Breaking the Full-Replica Barrier with a Federation of MoEs
  2. 2025Cluster Topology-Driven Placement of Experts Reduces Network Traffic in MoE Inference
  3. 2026CRAFT: Fine-Grained Cost-Aware Expert Replication For Efficient Mixture-of-Experts Serving
  4. 2026Function Over Form: Distributional Orthogonalization in Mixture-of-Experts with Replica Expert Mechanism
  5. 2022SMILE: Scaling Mixture-of-Experts with Efficient Bi-level Routing
  6. 2026Fast MoE Inference via Predictive Prefetching and Expert Replication
  7. 2026A Replicate-and-Quantize Strategy for Plug-and-Play Load Balancing of Sparse Mixture-of-Experts LLMs
  8. 2026Memory-Efficient Expert Routing for Distributed MoE Training