mlsanityData to 14 Oct 2026

Searching for Plans You Can Actually Build: A Realizability-Aware Full-Space Optimizer for MoE Training and Serving

arXiv 2607.18631 ↗First posted 21 Jul 2026cs.DCcited by 1

On the map: Mixture-of-experts → Efficient MoE training and inference systems

Abstract (arXiv)

Mixture-of-Experts (MoE) systems split a program's plan space in two: the space a cost model can rank, and the smaller space a real toolchain can actually build. Automatic optimizers rank the first and silently assume the two coincide -- so they can return a plan that is optimal on paper and impossible to emit. We present moefs, a realizability-aware full-space optimizer for MoE training and serving that makes deployment realizability a first-class search constraint. moefs closes a three-tier search over parallelism, schedule, and kernels; it emits both a Megatron training stack and an SGLang serving stack from a single plan; and it prices, rather than forbids, the realization overheads it measures. We evaluate across two hardware generations. On 2x RTX4090, the searched training plan edges the strongest hand-tuned baseline by +0.9% (and clears the 0.98x acceptance bar by +2.9%); on 8x H800, the searched serving plan matches the hand-tuned configuration at a 1.0304 throughput ratio. We hold failures to the same standard: on 8x H800 training, the searched plan is a computable, honest FAIL at 0.9338 of the best hand-tuned throughput, losing on a single schedule flag. All predictions are pre-registered in a frozen, artifact-hashed adjudication file before the H800 runs, and every outcome is reported as-is.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Efficient MoE training and inference systems, one of 12 directions in Mixture-of-experts. Builds on 21 of its 32 references that are in the map. First posted on arXiv 21 Jul 2026.

Builds on

21 of 32 references.

Closest ancestors · 11

Same direction, newest first

  1. 2026UltraEP: Unleash MoE Training and Inference on Rack-Scale Nodes with Near-Optimal Load Balancing2 Jun
  2. 2026UniEP: Unified Expert-Parallel MoE MegaKernel for LLM Training21 Apr
  3. 2025Training Report of TeleChat3-MoE30 Dec
  4. 2025UCCL-EP: Portable Expert-Parallel Communication22 Dec
  5. 2025FlashMoE: Fast Distributed MoE in a Single Kernel5 Jun
  6. 2025MegaScale-MoE: Large-Scale Communication-Efficient Training of Mixture-of-Experts Models in Production16 May
  7. 2025MoE-CAP: Benchmarking Cost, Accuracy and Performance of Sparse Mixture-of-Experts Systems16 May
  8. 2025Comet: Fine-grained Computation-communication Overlapping for Mixture-of-Experts27 Feb
  9. 2024Lancet: Accelerating Mixture-of-Experts Training via Whole Graph Computation-Communication Overlapping30 Apr
  10. 2022MegaBlocks: Efficient Sparse Training with Mixture-of-Experts29 Nov
  11. 2022Tutel: Adaptive Mixture-of-Experts at Scale7 Jun

Other references · 10

  1. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  2. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  3. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  4. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
  5. 2024Vidur: A Large-Scale Simulation Framework For LLM Inferencecited by 117
  6. 2022Galvatron: Efficient Transformer Training over Multiple GPUs Using Automatic Parallelismcited by 53
  7. 2023FlexMoE: Scaling Large-scale Sparse Pre-trained Model Training via Dynamic Device Placementcited by 53
  8. 2025Triton-distributed: Programming Overlapping Kernels on Distributed AI Systems with the Triton Compilercited by 25
  9. 2025TileLink: Generating Efficient Compute-Communication Overlapping Kernels using Tile-Centric Primitivescited by 24
  10. 2026DITRON: Distributed Multi-level Tiling Compiler for Parallel Tensor Programscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026ATTUNE: Agentic Throughput Tuning for LLM Trainingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026HAPMoE: Heterogeneity-Aware Automatic Parallelism Planning for Mixture-of-Experts Models Training
  2. 2026Holistic Scaling Laws for Optimal Mixture-of-Experts Architecture Optimization
  3. 2026Moebius: Serving Mixture-of-Expert Models with Seamless Runtime Parallelism Switch
  4. 2026FastEEP: Fast Intra-Instance Elastic Scaling for Expert-Parallel MoE Serving
  5. 2026PROBE: Co-Balancing Computation and Communication in MoE Inference via Real-Time Predictive Prefetching
  6. 2026Where Should Optimizer State Live? Tiered State Allocation for Memory-Efficient Mixture-of-Experts Training
  7. 2025MoE-Prism: Disentangling Monolithic Experts for Elastic MoE Services via Model-System Co-Designs
  8. 2026RaMP: Runtime-Aware Megakernel Polymorphism for Mixture-of-Experts