mlsanityData to 7 Oct 2026

RaMP: Runtime-Aware Megakernel Polymorphism for Mixture-of-Experts

arXiv 2604.26039 ↗First posted 28 Apr 2026cs.LG cs.AI cs.DCcited by 0

On the map: Mixture-of-experts → Efficient MoE training and inference systems

Abstract (arXiv)

The optimal kernel configuration for Mixture-of-Experts (MoE) inference depends on both batch size and the expert routing distribution, yet production systems dispatch from batch size alone, leaving 10-70% of kernel throughput unrealized. We present RaMP, a routing-aware dispatch framework. A performance-region analysis derives, from hardware constants alone, when each optimization helps, correctly predicting all 8 tested architectures, including 3 unseen. A four-parameter wave cost model selects the fastest configuration from the runtime expert histogram, achieving 0.93% mean regret versus exhaustive search, fitted from just 10-24 minutes of one-time profiling per model. Because the model depends only on CTA grid geometry, it is kernel-agnostic: applied to Alpha-MoE, it delivers 1.14x with no source modification. Paired with a co-designed CuTe DSL kernel exposing 134-268 polymorphic configurations, RaMP delivers 1.22x kernel speedup over static dispatch and 1.30x end-to-end speedup in vLLM serving over Triton, 1.41x over DeepGEMM, and 1.13x over FlashInfer CUTLASS.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Efficient MoE training and inference systems, one of 12 directions in Mixture-of-experts. Builds on 11 of its 17 references that are in the map. First posted on arXiv 28 Apr 2026.

Builds on

11 of 17 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025SonicMoE: Accelerating MoE with IO and Tile-aware Optimizations16 Dec
  2. 2024Scattered Mixture-of-Experts Implementation13 Mar
  3. 2022MegaBlocks: Efficient Sparse Training with Mixture-of-Experts29 Nov
  4. 2022Tutel: Adaptive Mixture-of-Experts at Scale7 Jun

Other references · 7

  1. 2025Qwen3 Technical Reportcited by 8,195
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  3. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  4. 2024OLMoE: Open Mixture-of-Experts Language Modelscited by 291
  5. 2025FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Servingcited by 248
  6. 2020Ansor: Generating High-Performance Tensor Programs for Deep Learningcited by 199
  7. 2014Quantifying performance bottlenecks of stencil computations using the Execution-Cache-Memory modelcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Decoding the Skew: Distribution-Aware MoE Inference with Adaptive Kernel Dispatch
  2. 2026APEX: Adaptive Expert Prefetching for Memory-Efficient Edge MoE Inference
  3. 2024ExpertFlow: Efficient Mixture-of-Experts Inference via Predictive Expert Caching and Token Scheduling
  4. 2026SpecPrefetch: Parameter-Efficient Expert Prefetching for Sparse MoE Foundation Models
  5. 2026EasyBalance: Cross-Layer Load Balancing in Distributed MoE Inference
  6. 2026ExpertPlex: A High-Goodput Disaggregated Serving System for MoE LLMs with Adaptive Persistent Kernels
  7. 2025ExpertFlow: Adaptive Expert Scheduling and Memory Coordination for Efficient MoE Inference
  8. 2026DECO: Sparse Mixture-of-Experts with Dense-Comparable Performance on End-Side Devices