mlsanityData to 14 Oct 2026

Displacement Is Not Direction: Evaluating Fidelity Metrics for Quantized LLM Deployment

arXiv 2606.19558 ↗First posted 17 Jun 2026cs.LG cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

Fidelity metrics, such as per-token KL divergence (KLD) against a high-precision reference, are often used in practice as low-cost proxies for benchmark quality. We test this practice on a 28-quant cohort of Qwen3.6-35B-A3B and a 41-quant cohort of Devstral-Small-2-24B, evaluated across a suite of downstream benchmarks. We find that KLD is strongly correlated with benchmark score over the full cohort (ρ=-0.72 on Qwen and ρ=-0.86 on Devstral, both with p<0.001). However, this relationship collapses to non-significance in the near-baseline silent zone (ρ=+0.00 on Qwen and ρ=-0.24, p=0.36, on Devstral). This collapse persists across 14 measurement variants, including different KLD aggregations, perplexity formulations, top-1 agreement, calibration corpora, and context lengths. At the per-prompt level, KLD has only weak failure-prediction power on code, with failed-vs-passed geometric-mean ratios in [1.08,1.22] across five models on LiveCodeBench, and fails as a cross-model router, achieving only 42.3%-49.4% accuracy on disagreement prompts. We trace the collapse to a structural decomposition: KLD primarily measures the volume of disagreement with the reference, with silent-zone composite ρ=+0.94 (p<0.001) on Qwen and +0.55 (p=0.03) on Devstral, while its relationship to the direction of those disagreements is weak and task-conditional.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 14 of its 21 references that are in the map. First posted on arXiv 17 Jun 2026.

Builds on

14 of 21 references.

References · 14

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  4. 2023Let's Verify Step by Stepcited by 3,789
  5. 2016Pointer Sentinel Mixture Modelscited by 3,096
  6. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  7. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  8. 2023Compressing LLMs: The Truth is Rarely Pure and Never Simplecited by 62
  9. 2024"Give Me BF16 or Give Me Death"? Accuracy-Performance Trade-Offs in LLM Quantizationcited by 34
  10. 2024MixLLM: LLM Quantization with Global Mixed-precision between Output-features and Highly-efficient System Designcited by 16
  11. 2020BitPruning: Learning Bitlengths for Aggressive and Accurate Quantizationcited by 14
  12. 2024Exploring the Trade-Offs: Quantization Methods, Task Difficulty, and Model Size in Large Language Models From Edge to Giantcited by 14
2 more
  1. 2025GSM8K-V: Can Vision Language Models Solve Grade School Math Word Problems in Visual Contextscited by 8
  2. 2023Divergent Token Metrics: Measuring degradation to prune away LLM components -- and optimize quantizationcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Certifying Compressed Language Models: An Audit and a Statistical Toolkitcited by 2
  2. 2026Tied Trit-Planes: Constraining PTQTP to a Uniform Nine-Level Quantizer, with a Persistent Folded Format for Disk-Streamed Mixture-of-Experts Servingcited by 0
  3. 2026How Divergence Becomes Decision Flips in Compressed Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Accuracy is Not Enough: A Divergence-Based Approach to Evaluate Fidelity Loss in Quantized LLMs
  2. 2026Quantization Undoes Alignment: Bias Emergence in Compressed LLMs Across Models and Precision Levels
  3. 2025Safety-Preserving PTQ via Contrastive Alignment Loss
  4. 2026Saturation Makes Quantization Error Additive: A Coverage Model with a Certificate
  5. 2026Statistical Inference and Quality Measures of KV Cache Quantisations Inspired by TurboQuant
  6. 2026The Joint Effect of Quantization and Sampling Temperature on LLM Safety Alignment: A Factorial Analysis
  7. 2026MXSens: Sensitivity-Aware Mixed-Precision Quantization for Efficient LLM Inference
  8. 2026Demystifying Numerical Instability in LLM Inference: Achieving Reproducible Inference for Mission-Critical Tasks with HEAL