mlsanityData to 14 Oct 2026

Beyond Scalar Sensitivity: Activation-Aware Mixed-Precision LLM Quantization with Cross-Layer Refinement

arXiv 2609.25916 ↗First posted 22 Sep 2026cs.LG stat.AP stat.MEcited by 1

Not on the map. Why?

Abstract (arXiv)

Mixed-precision weight quantization is commonly formulated as a Multiple-Choice Knapsack Problem (MCKP), yet existing solvers rely on scalar sensitivity proxies that collapse each weight matrix's Hessian into a single number and treat every module independently. We prove that even the optimal scalar proxy incurs multiplicative distortion up to √kappa(A)kappa(B) relative to the full activation-aware quadratic, where kappa(A) and kappa(B) denote the condition numbers of the input- and output-side Hessian factors. This bound varies from 101 to 1013 for typical LLM modules, making inter-module sensitivity ranking unreliable. To address these limitations, we propose Cross-layer Activation-aware Sensitivity Allocation (CASA), a two-phase method. In Stage 1, the scalar proxy is replaced by an activation-aware metric derived from the Kronecker-factored Hessian, reducing the MCKP to a form whose continuous relaxation admits a closed-form solution. In Stage 2, a cross-layer-aware local search evaluates bit-width updates using the end-to-end model loss. Experiments on multiple LLMs across different bit budgets show that CASA achieves lower perplexity than the latest scalar-proxy baselines, especially at ultra-low bit-widths (<3 bits per weight). Moreover, the performance gain in zero-shot accuracy tracks the per-model average condition-number over modules, confirming the distortion bound as a practical indicator of scalar-proxy failure.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 16 of its 23 references that are in the map. First posted on arXiv 22 Sep 2026.

Builds on

16 of 23 references.

References · 16

  1. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  2. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  3. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  4. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 443
  5. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 402
  6. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 322
  7. 2019HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networkscited by 261
  8. 2021Towards Mixed-Precision Quantization of Neural Networks via Constrained Optimizationcited by 61
  9. 2020FracBits: Mixed Precision Quantization via Fractional Bit-Widthscited by 55
  10. 2025Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantizationcited by 41
  11. 2025KVTuner: Sensitivity-Aware Layer-Wise Mixed-Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inferencecited by 40
  12. 2021OMPQ: Orthogonal Mixed Precision Quantizationcited by 39
4 more
  1. 2024The SCIP Optimization Suite 9.0cited by 37
  2. 2025SlimLLM: Accurate Structured Pruning for Large Language Modelscited by 18
  3. 2025Q-Palette: Fractional-Bit Quantizers Toward Optimal Bit Allocation for Efficient LLM Deploymentcited by 8
  4. 2025InfoQ: Mixed-Precision Quantization via Global Information Flowcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Quantization Error Is Spectrally Flat: A Single Random Probe Is a Calibrated, Data-Free Sensitivity Estimator, with Application to Budget-Targeted Mixed-Precision Quantizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Taming Sensitive Weights : Noise Perturbation Fine-tuning for Robust LLM Quantization
  2. 2026Beyond Outliers: A Data-Free Layer-wise Mixed-Precision Quantization Approach Driven by Numerical and Structural Dual-Sensitivity
  3. 2026Q-PACE: Dynamic Precision Allocation for Quantization-Aware Training
  4. 2026Technical Report: Activation Residual Hessian Quantization (ARHQ) for Low-Bit LLM Quantization
  5. 2026Task-Aware QUBO Allocation for Mixed-Precision Quantization
  6. 2024MixLLM: LLM Quantization with Global Mixed-precision between Output-features and Highly-efficient System Design
  7. 2026ScaleSweep: Accurate NVFP4 Post-Training Quantization of LLMs via Block Scale Initialization
  8. 2026MXSens: Sensitivity-Aware Mixed-Precision Quantization for Efficient LLM Inference