mlsanityData to 14 Oct 2026

Behavioral Capacity Certificates for Quantized Language Models

arXiv 2609.37887 ↗First posted 29 Sep 2026cs.LGcited by 0

On the map: Efficient LLM inference and serving → KV cache quantization also fits: Efficient reasoning and agent inference

Abstract (arXiv)

Activation and key-value cache precision change what a quantized language model computes without altering its stored weights. Direct weight-code bounds, however, assign identical complexity to deployments that behave differently and charge separately for weight codes that behave identically. Behavioral Capacity Certificates (BCC) charge for behavior using the aggregate prior mass of complete implementations---weights, scales, activation and cache rules---that induce the same bounded loss. When quantization merges implementations, this shared mass lowers the complexity penalty, and a break-even law determines when the saving survives the cost of validating it. BCC supports a three-step deployment workflow, and our experiments verify each step. First, a forward-only screen shortlists per-layer bit-widths by how often candidate perturbations preserve the reference predictions, with quality comparable to Hessian-guided selection at lower preprocessing cost. Second, margin-certified cells identify weights that can be pruned or sign-flipped without changing the deployed behavior: every permitted combination preserves all declared predictions, and on OLMoE-1B-7B and SmolLM2-1.7B, independent probes bound the probability that any permitted combination changes a prediction on new text. Third, BCC bounds the population loss of the deployed model, nonvacuously for complete decoders and more tightly than the compressed-code route. At equal cache memory, giving keys higher precision than values yields lower NLL and higher prediction agreement on GPT-2, Qwen2.5, and SmolLM2, together with a tighter complexity bound in the GPT-2 audit.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in KV cache quantization, one of 12 directions in Efficient LLM inference and serving. Builds on 18 of its 23 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

18 of 23 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Quantize What Counts: More for Keys, Less for Values20 Feb
  2. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache5 Feb

Other references · 16

  1. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  2. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  3. 2024Mixtral of Expertscited by 1,823
  4. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  5. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  6. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 533
  7. 2017Sharp Minima Can Generalize For Deep Netscited by 466
  8. 2018Stronger generalization bounds for deep nets via a compression approachcited by 307
  9. 2019HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networkscited by 261
  10. 2018Non-Vacuous Generalization Bounds at the ImageNet Scale: A PAC-Bayesian Compression Approachcited by 126
  11. 2022PAC-Bayes Compression Bounds So Tight That They Can Explain Generalizationcited by 76
  12. 2023Non-Vacuous Generalization Bounds for Large Language Modelscited by 53
4 more
  1. 2025SpindleKV: A Novel KV Cache Reduction Method Balancing Both Shallow and Deep Layerscited by 9
  2. 2023Exploring the Complexity of Deep Neural Networks through Functional Equivalencecited by 4
  3. 2025Symmetries in PAC-Bayesian Learningcited by 1
  4. 2026Breaking the 1.58-bit Barrier for Ternary LLMscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026On the Expressive Power of Weight Quantization in Large Language Models
  2. 2026Reliability Scaling Laws for Quantized Large Language Models
  3. 2026Accuracy is Not Enough: A Divergence-Based Approach to Evaluate Fidelity Loss in Quantized LLMs
  4. 2026Quantization Undoes Alignment: Bias Emergence in Compressed LLMs Across Models and Precision Levels
  5. 2026The Structure of Quantization Damage in LLMs: Why the Next Bit Should Be Spent Globally
  6. 2026RAMP: Reinforcement Adaptive Mixed Precision Quantization for Efficient On Device LLM Inference
  7. 2026Quantization Degradation in Large Language Models: A Signal-Noise Perspective
  8. 2024Exploring the Trade-Offs: Quantization Methods, Task Difficulty, and Model Size in Large Language Models From Edge to Giant