What You Sign Is Not What You Run: The Verifiability Gap in Quantized LLM Supply Chains
Not on the map. Why?
Abstract (arXiv)
Model signing authenticates the full-precision weights W an LLM is released with, yet users run a quantized artifact Q whose producing transformation is not uniquely determined under current disclosure. We separate the non-environment inputs from the execution environment E and measure a quantity that, to our knowledge, has not been measured before: the Independent Verifiability Rate (IVR), the fraction of a popularity-sampled GGUF ecosystem that a third party can re-derive bit-for-bit. Across the 200 most-downloaded Hugging Face GGUF repositories (198 in scope), no artifact or model card records the build environment, so under a recorded-inputs protocol the IVR is 0/198. Yet the artifacts are not irreproducible. On four in-sample repositories that disclose their inputs, the card's information alone reproduces no Q4_K_M file, but two coordinates the card omits - the build's floating-point-contraction setting and, for hybrid models, the per-tensor quantization recipe - re-derive 7/8 target-quant files bit-for-bit on x86 (5/8 on arm64); the fp16 re-derives from the provider's checkpoint, but the calibration imatrix must be shipped and hash-bound, since recomputing it did not match. An older release additionally pins the instruction set and did not re-derive. Reproducing the divergence reported with llama.cpp PR #25353, we show that E is a necessary coordinate and that one compiler setting yields a unique output across the tested platforms at negligible quality cost (+0.06% perplexity). As a security characterization, benign floating-point non-determinism separates from uniformly placed nibble-flip tampering above a detection floor that grows with the environments a verifier must allow, but a targeted embedding edit evades the statistic, so the proposed protocol rests on exact re-derivation; adversarial-fp16 attacks lie outside it. We release the audit data and a re-runnable harness.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 45 references that are in the map. First posted on arXiv 14 Oct 2026.
Builds on
18 of 45 references.
References · 18
- 2016Pointer Sentinel Mixture Modelscited by 3,096
- 2024Exploiting LLM Quantizationcited by 65
- 2023An Empirical Study of Pre-Trained Model Reuse in the Hugging Face Deep Learning Model Registrycited by 47
- 2021Quantization Backdoors to Deep Learning Commercial Frameworkscited by 29
- 2021Qu-ANTI-zation: Exploiting Quantization Artifacts for Achieving Adversarial Outcomescited by 19
- 2024Nearest is Not Dearest: Towards Practical Defense against Quantization-conditioned Backdoor Attackscited by 18
- 2025TOPLOC: A Locality Sensitive Hashing Scheme for Trustless Verifiable Inferencecited by 17
- 2024What's documented in AI? Systematic Analysis of 32K AI Model Cardscited by 11
- 2021Reproducible Builds: Increasing the Integrity of Software Supply Chainscited by 7
- 2024PrisonBreak: Jailbreaking Large Language Models with at Most Twenty-Five Targeted Bit-flipscited by 5
- 2025Rounding-Guided Backdoor Injection in Deep Learning Model Quantizationcited by 3
- 2025Attestable Builds: Compiling Verifiable Binaries on Untrusted Systems using Trusted Execution Environmentscited by 2
6 more
- 2026Inference-Time Backdoors via Chat Templates: From LLM Supply Chains to Agentic System Compromisecited by 2
- 2026Attesting LLM Pipelines: Enforcing Verifiable Training and Release Claimscited by 2
- 2026Certifying Compressed Language Models: An Audit and a Statistical Toolkitcited by 2
- 2026Broken on Arrival: Silently Defective LLM Artifacts in Public Model Registries and How to Catch Themcited by 2
- 2018Automated Localization for Unreproducible Buildscited by 1
- 2026BadTemplate: A Training-Free Backdoor Attack via Chat Template Against Large Language Modelscited by 1
Built on by
Not cited yet.