mlsanityData to 7 Oct 2026

Quantization Error Is Spectrally Flat: A Single Random Probe Is a Calibrated, Data-Free Sensitivity Estimator, with Application to Budget-Targeted Mixed-Precision Quantization

arXiv 2609.33923 ↗First posted 27 Sep 2026cs.CL cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

A single random Gaussian probe gives an unbiased estimate of the squared Frobenius norm of a layer's quantization error. The estimator is well-behaved because round-to-nearest error is spectrally flat. Across 1,683 tensors from a 35B MoE and a 9B dense model, effective dimensionality is 0.93 to 0.96 times the i.i.d. noise value of the same shape, and on the MoE the median is unchanged from 2-bit to 8-bit. The probe coefficient of variation is predictable from tensor shape. One probe measures per-tensor sensitivity to within 4 to 7%; twenty probes reach 1.3 to 1.4%.RAM applies the propagated form of this estimator to budget-targeted mixed-precision quantization with no calibration data. Gaussian probes carrying the network's own input statistics score every tensor at six bit-widths. A knapsack solver allocates bits under an exact byte budget, with guardrails against catastrophic 2-bit assignments. One probe pass serves any budget. Isolated and propagated scores rank tensors independently on Qwen3.5-35B-A3B (Spearman -0.01), yet the propagated probe rank-correlates 0.81 to 0.83 with the GPTQ layer objective from real activations, while the isolated estimator is uncorrelated with it. That objective is the wrong allocation target: at matched bytes on Qwen3.8-27B, a block-output probe beats a vendor IQ3_M mix and an oracle that allocates from the real-activation objective. On Qwen3-8B the propagated probe ties HAWQ-V2 at matched bytes. Across seven architectures from 8B to 122B, with probe timing up to a 400B model in nine minutes on one workstation, RAM reaches 3.5 to 13.6% lower median WikiText-2 perplexity than size-comparable uniform 4-bit builds on the tested MoE models. (Black Sheep Ai baa.ai)

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 25 of its 40 references that are in the map. First posted on arXiv 27 Sep 2026.

Builds on

25 of 40 references.

References · 25

  1. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  2. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  3. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  4. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
  5. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
  6. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 397
  7. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 387
  8. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 359
  9. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 285
  10. 2019HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networkscited by 250
  11. 2024Extreme Compression of Large Language Models via Additive Quantizationcited by 223
  12. 2024SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Modelscited by 61
13 more
  1. 2024LQER: Low-Rank Quantization Error Reconstruction for LLMscited by 39
  2. 2024Mixture Compressor for Mixture-of-Experts LLMs Gains Morecited by 38
  3. 2024Pushing the Limits of Large Language Model Quantization via the Linearity Theoremcited by 20
  4. 2024EasyQuant: An Efficient Data-free Quantization Algorithm for LLMscited by 17
  5. 2025KurTail : Kurtosis-based LLM Quantizationcited by 12
  6. 2024Cherry on Top: Parameter Heterogeneity and Quantization in Large Language Modelscited by 9
  7. 2026Beyond Outliers: A Data-Free Layer-wise Mixed-Precision Quantization Approach Driven by Numerical and Structural Dual-Sensitivitycited by 5
  8. 2026AlphaQ: Calibration-Free Bit Allocation for Mixture-of-Experts Quantizationcited by 4
  9. 2026Q-Strata: Hierarchical Bit Allocation for Mixed-Precision Quantization of Mixture-of-Experts LLMscited by 2
  10. 2026GAMMA: Global Bit Allocation for Mixed-Precision Models under Arbitrary Budgetscited by 1
  11. 2026MixQuant: Adaptive Mixed-Precision Quantization for Large Language Modelscited by 1
  12. 2026Fidelity Is Not Safety: Gently-Compressed LLMs Pass Every Data-Free Quality Guard Yet Invent Procedure Steps in Agentic Executioncited by 1
  13. 2026Beyond Scalar Sensitivity: Activation-Aware Mixed-Precision LLM Quantization with Cross-Layer Refinementcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Layerwise Error Attribution for Fast and Robust Mixed-Precision Post-Training Quantization
  2. 2024Taming Sensitive Weights : Noise Perturbation Fine-tuning for Robust LLM Quantization
  3. 2026ScaleSweep: Accurate NVFP4 Post-Training Quantization of LLMs via Block Scale Initialization
  4. 2026Provable Quantization with Randomized Hadamard Transform
  5. 2026MXSens: Sensitivity-Aware Mixed-Precision Quantization for Efficient LLM Inference
  6. 2026TORQUE: Optimizing What (not) to Quantize Before and After Rotation
  7. 2024MPQ-Diff: Mixed Precision Quantization for Diffusion Models
  8. 2025Elucidating the Design Space of FP4 training