Kernel Contracts: A Specification Language for ML Kernel Correctness Across Heterogeneous Silicon
Not on the map. Why?
Abstract (arXiv)
Every ML kernel ships with an implicit contract about what it computes. People rarely write the contract down. When two kernels disagree -- when a matmul on AMD produces a different gradient than the same matmul on NVIDIA, when a fused attention kernel silently downcasts an accumulator, when an out-of-bounds access returns zero on one stack and garbage on another -- there is no formal artifact to arbitrate the dispute. Recent empirical work has measured the gap across silicon platforms, but none of it specifies the contract being violated. We present a specification language for kernel contracts. A contract has eight parts: identifier, scope, precondition, postcondition, tolerance, reference oracle, measurement protocol, and violation signature. We use it to state twelve contract classes covering precision, ordering, compiler-induced, and exceptional-value failure modes, each grounded in published empirical evidence. We require a three-state calibration: every contract must admit at least one reference-conforming implementation and at least one contract-violating implementation that passes basic functional tests. We apply the framework to three documented incidents -- Huawei Ascend silent precision coercion, Sakana AI CUDA Engineer reward hacking, AMD out-of-bounds silent acceptance -- and show that each informal diagnosis maps to a specific contract violation with a measurable signature. A kernel contract suite is a normative reference against which conformance can be graded, in the way that ISASecure grades industrial control systems against IEC 62443.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 11 of its 40 references that are in the map. First posted on arXiv 23 Apr 2026.
Builds on
11 of 40 references.
References · 11
- 2024FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisioncited by 507
- 2025Towards Robust Agentic CUDA Kernel Benchmarking, Verification, and Optimizationcited by 52
- 2025Understanding and Mitigating Numerical Sources of Nondeterminism in LLM Inferencecited by 47
- 2025MultiKernelBench: A Multi-Platform Benchmark for Kernel Generationcited by 35
- 2024Impacts of floating-point non-associativity on reproducibility for HPC and deep learning applicationscited by 24
- 2025Understanding Silent Data Corruption in LLM Trainingcited by 13
- 2025Equivalence Checking of ML GPU Kernelscited by 7
- 2024Testing GPU Numerics: Finding Numerical Differences Between NVIDIA and AMD GPUscited by 4
- 2024Exo 2: Growing a Scheduling Languagecited by 3
- 2026DASH: Deterministic Attention Scheduling for High-throughput Reproducible LLM Trainingcited by 3
- 2026Reward Hacking as Equilibrium under Finite Evaluationcited by 3
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026The Integer Alibi: Localizing Cross-Kernel Divergence in INT8-Quantized LLM Inferencecited by 2
- 2026A Contract-Grade Verifier for LLM-Generated GPU Kernels, and a Native Blackwell Backward for the Gated-Linear-Recurrence Familycited by 0
- 2026Energy-efficient operation of neural operators for virtual sensingcited by 0
- 2026Energy-efficient operation of neural operators for virtual sensing24 Sep 2026
- 2026The Integer Alibi: Localizing Cross-Kernel Divergence in INT8-Quantized LLM Inference13 Aug 2026
- 2026A Contract-Grade Verifier for LLM-Generated GPU Kernels, and a Native Blackwell Backward for the Gated-Linear-Recurrence Family13 Aug 2026
Similar, not cited
Close in wording, no citation either way
- 2026FastKernels: Benchmarking GPU Kernel Generation in Production
- 2026KernelBench-Verified: Do LLM-Generated Kernels Actually Beat PyTorch?
- 2026KernelGenBench: Can LLMs and Agents Write Efficient Kernels Across Operator Sources and Hardware Platforms?
- 2026Are LLM-Generated GPU Kernels Production-Ready? A Trace-Driven Benchmark and Optimization Agent
- 2026KernelCraft: Benchmarking for Agentic Close-to-Metal Kernel Generation on Emerging Hardware
- 2025ProofWright: Towards Agentic Formal Verification of CUDA
- 2026Improving Efficiency of GPU Kernel Optimization Agents using a Domain-Specific Language and Speed-of-Light Guidance
- 2026Spec Sheets Are Not Kernels: An ISA- and Source-Level Audit of INT8 Availability on NVIDIA Blackwell Ultra