# Kernel Contracts: A Specification Language for ML Kernel Correctness Across Heterogeneous Silicon

arXiv: https://arxiv.org/abs/2604.22032 · HTML: https://mlsanity.com/paper/2604.22032/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 40 references that are in the map.
- First posted on arXiv 23 Apr 2026.

## Builds on (most cited first, 11 of 11)

- [FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision](https://mlsanity.com/paper/2407.08608/) (2024)
- [Towards Robust Agentic CUDA Kernel Benchmarking, Verification, and Optimization](https://mlsanity.com/paper/2509.14279/) (2025)
- [Understanding and Mitigating Numerical Sources of Nondeterminism in LLM Inference](https://mlsanity.com/paper/2506.09501/) (2025)
- [MultiKernelBench: A Multi-Platform Benchmark for Kernel Generation](https://mlsanity.com/paper/2507.17773/) (2025)
- [Impacts of floating-point non-associativity on reproducibility for HPC and deep learning applications](https://mlsanity.com/paper/2408.05148/) (2024)
- [Understanding Silent Data Corruption in LLM Training](https://mlsanity.com/paper/2502.12340/) (2025)
- [Equivalence Checking of ML GPU Kernels](https://mlsanity.com/paper/2511.12638/) (2025)
- [Testing GPU Numerics: Finding Numerical Differences Between NVIDIA and AMD GPUs](https://mlsanity.com/paper/2410.09172/) (2024)
- [Reward Hacking as Equilibrium under Finite Evaluation](https://mlsanity.com/paper/2603.28063/) (2026)
- [DASH: Deterministic Attention Scheduling for High-throughput Reproducible LLM Training](https://mlsanity.com/paper/2601.21824/) (2026)
- [Exo 2: Growing a Scheduling Language](https://mlsanity.com/paper/2411.07211/) (2024)

## Built on by (most cited first, 3 of 3)

- [The Integer Alibi: Localizing Cross-Kernel Divergence in INT8-Quantized LLM Inference](https://mlsanity.com/paper/2608.13756/) (2026)
- [Energy-efficient operation of neural operators for virtual sensing](https://mlsanity.com/paper/2609.30580/) (2026)
- [A Contract-Grade Verifier for LLM-Generated GPU Kernels, and a Native Blackwell Backward for the Gated-Linear-Recurrence Family](https://mlsanity.com/paper/2608.12700/) (2026)

## Abstract (arXiv)

Every ML kernel ships with an implicit contract about what it computes. People rarely write the contract down. When two kernels disagree -- when a matmul on AMD produces a different gradient than the same matmul on NVIDIA, when a fused attention kernel silently downcasts an accumulator, when an out-of-bounds access returns zero on one stack and garbage on another -- there is no formal artifact to arbitrate the dispute. Recent empirical work has measured the gap across silicon platforms, but none of it specifies the contract being violated. We present a specification language for kernel contracts. A contract has eight parts: identifier, scope, precondition, postcondition, tolerance, reference oracle, measurement protocol, and violation signature. We use it to state twelve contract classes covering precision, ordering, compiler-induced, and exceptional-value failure modes, each grounded in published empirical evidence. We require a three-state calibration: every contract must admit at least one reference-conforming implementation and at least one contract-violating implementation that passes basic functional tests. We apply the framework to three documented incidents -- Huawei Ascend silent precision coercion, Sakana AI CUDA Engineer reward hacking, AMD out-of-bounds silent acceptance -- and show that each informal diagnosis maps to a specific contract violation with a measurable signature. A kernel contract suite is a normative reference against which conformance can be graded, in the way that ISASecure grades industrial control systems against IEC 62443.