# What You Sign Is Not What You Run: The Verifiability Gap in Quantized LLM Supply Chains

arXiv: https://arxiv.org/abs/2610.19498 · HTML: https://mlsanity.com/paper/2610.19498/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 45 references that are in the map.
- First posted on arXiv 14 Oct 2026.

## Builds on (most cited first, 18 of 18)

- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [Exploiting LLM Quantization](https://mlsanity.com/paper/2405.18137/) (2024)
- [An Empirical Study of Pre-Trained Model Reuse in the Hugging Face Deep Learning Model Registry](https://mlsanity.com/paper/2303.02552/) (2023)
- [Quantization Backdoors to Deep Learning Commercial Frameworks](https://mlsanity.com/paper/2108.09187/) (2021)
- [Qu-ANTI-zation: Exploiting Quantization Artifacts for Achieving Adversarial Outcomes](https://mlsanity.com/paper/2110.13541/) (2021)
- [Nearest is Not Dearest: Towards Practical Defense against Quantization-conditioned Backdoor Attacks](https://mlsanity.com/paper/2405.12725/) (2024)
- [TOPLOC: A Locality Sensitive Hashing Scheme for Trustless Verifiable Inference](https://mlsanity.com/paper/2501.16007/) (2025)
- [What's documented in AI? Systematic Analysis of 32K AI Model Cards](https://mlsanity.com/paper/2402.05160/) (2024)
- [Reproducible Builds: Increasing the Integrity of Software Supply Chains](https://mlsanity.com/paper/2104.06020/) (2021)
- [PrisonBreak: Jailbreaking Large Language Models with at Most Twenty-Five Targeted Bit-flips](https://mlsanity.com/paper/2412.07192/) (2024)
- [Rounding-Guided Backdoor Injection in Deep Learning Model Quantization](https://mlsanity.com/paper/2510.09647/) (2025)
- [Attestable Builds: Compiling Verifiable Binaries on Untrusted Systems using Trusted Execution Environments](https://mlsanity.com/paper/2505.02521/) (2025)
- [Inference-Time Backdoors via Chat Templates: From LLM Supply Chains to Agentic System Compromise](https://mlsanity.com/paper/2602.04653/) (2026)
- [Attesting LLM Pipelines: Enforcing Verifiable Training and Release Claims](https://mlsanity.com/paper/2603.28988/) (2026)
- [Certifying Compressed Language Models: An Audit and a Statistical Toolkit](https://mlsanity.com/paper/2608.15046/) (2026)
- [Broken on Arrival: Silently Defective LLM Artifacts in Public Model Registries and How to Catch Them](https://mlsanity.com/paper/2609.05881/) (2026)
- [Automated Localization for Unreproducible Builds](https://mlsanity.com/paper/1803.06766/) (2018)
- [BadTemplate: A Training-Free Backdoor Attack via Chat Template Against Large Language Models](https://mlsanity.com/paper/2602.05401/) (2026)

## Abstract (arXiv)

Model signing authenticates the full-precision weights W an LLM is released with, yet users run a quantized artifact Q whose producing transformation is not uniquely determined under current disclosure. We separate the non-environment inputs from the execution environment E and measure a quantity that, to our knowledge, has not been measured before: the Independent Verifiability Rate (IVR), the fraction of a popularity-sampled GGUF ecosystem that a third party can re-derive bit-for-bit. Across the 200 most-downloaded Hugging Face GGUF repositories (198 in scope), no artifact or model card records the build environment, so under a recorded-inputs protocol the IVR is 0/198. Yet the artifacts are not irreproducible. On four in-sample repositories that disclose their inputs, the card's information alone reproduces no Q4_K_M file, but two coordinates the card omits - the build's floating-point-contraction setting and, for hybrid models, the per-tensor quantization recipe - re-derive 7/8 target-quant files bit-for-bit on x86 (5/8 on arm64); the fp16 re-derives from the provider's checkpoint, but the calibration imatrix must be shipped and hash-bound, since recomputing it did not match. An older release additionally pins the instruction set and did not re-derive. Reproducing the divergence reported with llama.cpp PR #25353, we show that E is a necessary coordinate and that one compiler setting yields a unique output across the tested platforms at negligible quality cost (+0.06% perplexity). As a security characterization, benign floating-point non-determinism separates from uniformly placed nibble-flip tampering above a detection floor that grows with the environments a verifier must allow, but a targeted embedding edit evades the statistic, so the proposed protocol rests on exact re-derivation; adversarial-fp16 attacks lie outside it. We release the audit data and a re-runnable harness.