mlsanityData to 14 Oct 2026

MPK: A Compiler and Runtime for Mega-Kernelizing Tensor Programs

arXiv 2512.22219 ↗First posted 22 Dec 2025cs.DC cs.LG cs.PLcited by 15

Not on the map. Why?

Abstract (arXiv)

We introduce Mirage Persistent Kernel (MPK), the first compiler and runtime system that automatically transforms multi-GPU model inference into a single high-performance mega-kernel. MPK introduces an SM-level graph representation that captures data dependencies at the granularity of individual streaming multiprocessors (SMs), enabling cross-operator software pipelining, revfine-grained overlap of computation and communication, and other optimizations that are infeasible under the conventional kernel-per-operator execution model. The MPK compiler lowers tensor programs into optimized SM-level task graphs and generates fast CUDA implementations for each task, while the MPK in-kernel parallel runtime executes these tasks within a single persistent mega-kernel using decentralized scheduling across SMs. Together, these components provide end-to-end kernel fusion with minimal developer effort, while preserving the flexibility of existing programming models. Our evaluation shows that MPK significantly outperforms existing kernel-per-operator LLM serving systems, achieving up to 1.7× lower end-to-end inference latency and pushing LLM inference performance close to the limits of the underlying hardware. MPK is publicly available at https://github.com/mirage-project/mirage.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 14 of its 47 references that are in the map. First posted on arXiv 22 Dec 2025.

Builds on

14 of 47 references.

References · 14

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  2. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  3. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  4. 2025FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Servingcited by 258
  5. 2020Ansor: Generating High-Performance Tensor Programs for Deep Learningcited by 201
  6. 2018Learning to Optimize Tensor Programscited by 140
  7. 2024Mirage: A Multi-Level Superoptimizer for Tensor Programscited by 41
  8. 2022TensorIR: An Abstraction for Automatic Tensorized Program Optimizationcited by 38
  9. 2024ThunderKittens: Simple, Fast, and Adorable AI Kernelscited by 36
  10. 2021Equality Saturation for Tensor Graph Superoptimizationcited by 27
  11. 2025Triton-distributed: Programming Overlapping Kernels on Distributed AI Systems with the Triton Compilercited by 25
  12. 2025TokenWeave: Efficient Compute-Communication Overlap for Distributed LLM Inferencecited by 14
2 more
  1. 2025ParallelKittens: Systematic and Practical Simplification of Multi-GPU AI Kernelscited by 12
  2. 2024Optimal Kernel Orchestration for Tensor Programs with Korchcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 15.

15 papers.

Sort:

By area:

  1. 2026Event Tensor: A Unified Abstraction for Compiling Dynamic Megakernelcited by 4
  2. 2026Ada-MK: Adaptive MegaKernel Optimization via Automated DAG-based Search for LLM Inferencecited by 1
  3. 2026Vortex: Efficient and Programmable Sparse Attention Serving for AI Agentscited by 1
  4. 2026ExpertPlex: A High-Goodput Disaggregated Serving System for MoE LLMs with Adaptive Persistent Kernelscited by 1
  5. 2026PhyAI: Real-Time Physical AI at the Edge, Scalable Rollouts in the Cloudcited by 1
  6. 2026Ave: Guiding Agentic GPU Optimization Using Data-Flow Invariantscited by 0
  7. 2026AutoMegaKernel: A Statically-Checked Agent Harness for Self-Retargeting Megakernel Synthesiscited by 0
  8. 2026Measuring and Reducing WebGPU Dispatch Overhead for LLM Inferencecited by 0
  9. 2026Ready Cohorts: Bounding GPU Opportunity and Avoiding Host Round Trips in LLM-Agent Controlcited by 0
  10. 2026mKernel: Fast Multi-GPU, Multi-Node Fused Kernelscited by 0
  11. 2026Empowering Hybrid Attention Models on NPUscited by 0
  12. 2026Mixture-of-Kittens: MoE Megakernel for NVL72scited by 0
3 more
  1. 2026Purlin: Separating Orchestration from the Datapath of Collectivescited by 0
  2. 2026Trident: Unifying Guarded Dispatch and Host Execution for PyTorch Triton Workloadscited by 0
  3. 2026RESOLVE: Language-Agnostic Validation of GPU Kernels Through Testing, Reduction, and Proofcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026FastKernels: Benchmarking GPU Kernel Generation in Production
  2. 2026KernelBench-Verified: Do LLM-Generated Kernels Actually Beat PyTorch?
  3. 2026KernelOPT: Dispatch-Aware Agentic Search for GPU Kernel Optimization
  4. 2025Static Batching of Irregular Workloads on GPUs: Framework and Application to Efficient MoE Model Inference
  5. 2026Automated Tensor Scheduling for Hybrid CPU-GPU LLM Inference on Consumer Devices
  6. 2026Are LLM-Generated GPU Kernels Production-Ready? A Trace-Driven Benchmark and Optimization Agent
  7. 2026TCL: Enabling Fast and Efficient Cross-Hardware Tensor Program Optimization via Continual Learning
  8. 2026KernelBenchX: A Comprehensive Benchmark for Evaluating LLM-Generated GPU Kernels