mlsanityData to 14 Oct 2026

Beyond Outliers: A Data-Free Layer-wise Mixed-Precision Quantization Approach Driven by Numerical and Structural Dual-Sensitivity

arXiv 2603.17354 ↗First posted 18 Mar 2026cs.LG cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

Layer-wise mixed-precision quantization (LMPQ) enables effective compression under extreme low-bit settings by allocating higher precision to sensitive layers. However, existing methods typically treat all intra-layer weight modules uniformly and rely on a single numerical property when estimating sensitivity, overlooking their distinct operational roles and structural characteristics. To address this, we propose NSDS, a novel calibration-free LMPQ framework driven by Numerical and Structural Dual-Sensitivity. Specifically, it first mechanistically decomposes each layer into distinct operational roles and quantifies their sensitivity from both numerical and structural perspectives. These dual-aspect scores are then aggregated into a unified layer-wise metric through a robust aggregation scheme based on MAD-Sigmoid and Soft-OR to guide bit allocation. Extensive experiments demonstrate that NSDS consistently achieves superior performance compared to various baselines across diverse models and downstream tasks, without relying on any calibration data.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 30 of its 48 references that are in the map. First posted on arXiv 18 Mar 2026.

Builds on

30 of 48 references.

References · 30

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  4. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  5. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  6. 2016Pointer Sentinel Mixture Modelscited by 3,096
  7. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  8. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  9. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  10. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  11. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  12. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
18 more
  1. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 443
  2. 2019HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networkscited by 261
  3. 2023On the Impact of Calibration Data in Post-training Quantization and Pruningcited by 58
  4. 2025SwS: Self-aware Weakness-driven Problem Synthesis in Reinforcement Learning for LLM Reasoningcited by 42
  5. 2020Search What You Want: Barrier Panelty NAS for Mixed Precision Quantizationcited by 32
  6. 2025TreeReview: A Dynamic Tree of Questions Framework for Deep and Efficient LLM-based Scientific Peer Reviewcited by 31
  7. 2025Chain-of-Reasoning: Towards Unified Mathematical Reasoning in Large Language Models via a Multi-Paradigm Perspectivecited by 29
  8. 2022How to Dissect a Muppet: The Structure of Transformer Embedding Spacescited by 20
  9. 2024Layer-Wise Quantization: A Pragmatic and Effective Method for Quantizing LLMs Beyond Integer Bit-Levelscited by 16
  10. 2024UNComp: Can Matrix Entropy Uncover Sparsity? -- A Compressor Design from an Uncertainty-Aware Perspectivecited by 15
  11. 2025Towards Superior Quantization Accuracy: A Layer-sensitive Approachcited by 8
  12. 2025GuiLoMo: Allocating Expert Number and Rank for LoRA-MoE via Bilevel Optimization with GuidedSelection Vectorscited by 6
  13. 2025AlignX: Advancing Multilingual Large Language Models with Multilingual Representation Alignmentcited by 6
  14. 2023MixQuant: Mixed Precision Quantization with a Bit-width Optimization Searchcited by 4
  15. 2024LSAQ: Layer-Specific Adaptive Quantization for Large Language Model Deploymentcited by 4
  16. 2025Universality of Layer-Level Entropy-Weighted Quantization Beyond Model Architecture and Sizecited by 4
  17. 2026Training LLMs for Divide-and-Conquer Reasoning Elevates Test-Time Scalabilitycited by 4
  18. 2025Intrinsic Structure as a Proxy for Saliency: SVD-Based Weight Preservation for Mixed-Precision Quantization in Large Language Modelscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026OScaR: The Occam's Razor for Extreme KV Cache Quantization in LLMs and Beyondcited by 5
  2. 2026Massive Activations in Hybrid Linear Attention Large Language Models: Pre-Attention Spikes and Inter-Spike Plateauscited by 2
  3. 2026All for 1-Bit: Towards Genuine 1-Bit Post-Training Quantization for LLMscited by 2
  4. 2026Efficient Reasoning Exploration via State-Conditioned Latent Steering with Progress Guidancecited by 0
  5. 2026Quantization Error Is Spectrally Flat: A Single Random Probe Is a Calibrated, Data-Free Sensitivity Estimator, with Application to Budget-Targeted Mixed-Precision Quantizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Layerwise Error Attribution for Fast and Robust Mixed-Precision Post-Training Quantization
  2. 2026MXSens: Sensitivity-Aware Mixed-Precision Quantization for Efficient LLM Inference
  3. 2021Data-free mixed-precision quantization using novel sensitivity metric
  4. 2026MixQuant: Adaptive Mixed-Precision Quantization for Large Language Models
  5. 2024MixLLM: LLM Quantization with Global Mixed-precision between Output-features and Highly-efficient System Design
  6. 2026Q-PACE: Dynamic Precision Allocation for Quantization-Aware Training
  7. 2026Task-Aware QUBO Allocation for Mixed-Precision Quantization
  8. 2021BMPQ: Bit-Gradient Sensitivity Driven Mixed-Precision Quantization of DNNs from Scratch