MusaCoder: Native GPU Kernel Generation with Full-Stack Training on Moore Threads GPU
Not on the map. Why?
Abstract (arXiv)
Native GPU kernel generation turns high-level tensor programs into executable, efficient low-level code. Existing Large Language Models (LLMs) struggle with this task, while execution-based reinforcement learning suffers from sparse rewards, reward hacking, and training instability. We present MusaCoder, a full-stack training framework for native GPU kernel generation on CUDA and MUSA backends. MusaCoder combines progressive kernel-oriented data synthesis, diversity-preserving rejection fine-tuning, and execution-feedback Reinforcement Learning (RL) through MooreEval, a distributed verifier and reward environment. To stabilize RL, MusaCoder introduces PrimeEcho for first-turn-anchored multi-turn rewards, Buffered Dynamic Retry for recovering signals from all-failed hard samples, and MirrorPop for off-policy sequence filtering. Experiments on KernelBench and a MUSA-ported variant show that MusaCoder outperforms strong open-source and proprietary baselines in both correctness and empirical speedup, with the 9B model matching or exceeding frontier closed-source models and the 27B model establishing a new state of the art. These results demonstrate not only the effectiveness of full-stack execution-feedback training for native kernel generation, but also the capability of Moore Threads GPUs to support the complete LLM post-training stack, providing a practical foundation for large-model training and optimization on emerging accelerators.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Jun 2026.
Builds on
0 of 0 references.
No references here yet.
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026KernelZero: Co-Evolving Proposer and Coder for Continuously Improved GPU Kernel Generation
- 2026Fine-Tuning GPT-5 for GPU Kernel Generation
- 2026From Tokens to Regions: CUDA-Sensitive Instruction Tuning for GPU Kernel Generation
- 2026FastKernels: Benchmarking GPU Kernel Generation in Production
- 2026KernelBench-Verified: Do LLM-Generated Kernels Actually Beat PyTorch?
- 2025Kevin: Multi-Turn RL for Generating CUDA Kernels
- 2026A Two-Stage GPU Kernel Tuner Combining Semantic Refactoring and Search-Based Optimization
- 2026AMDKernelVault: Large-Scale Datasets and Agentic Training for AMD GPU Kernel Optimization