mlsanityData to 14 Oct 2026

CALM: A CKA-Guided Adaptive Layer-Wise Modularization Framework for LLM Quantization

arXiv 2512.16282 ↗First posted 18 Dec 2025cs.LG cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Current mainstream post-training quantization methods for large language models typically apply a uniform quantization strategy across all network layers, overlooking the substantial differences in algorithmic suitability among layers. To address this limitation, we propose CALM (A CKA-guided Adaptive Layer-wise Modularization)a fine-tuning-free, plug-and-play framework for algorithmic heterogeneous quantization. CALM independently evaluates multiple PTQ algorithms on each layer and employs Linear Centered Kernel Alignment (CKA) as a metric to automatically select the optimal quantization strategy per layer. The individually optimized strategies are then integrated to construct a hybrid quantized model. Experiments demonstrate that our approach consistently outperforms both uniform quantization baselines and state-of-the-art mixed-precision methods across mainstream LLMsincluding LLaMA and Qwenin terms of perplexity (PPL) and downstream task performance.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 15 of its 19 references that are in the map. First posted on arXiv 18 Dec 2025.

Builds on

15 of 19 references.

References · 15

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  2. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  3. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  4. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  5. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 675
  6. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
  7. 2019Similarity-Preserving Knowledge Distillationcited by 516
  8. 2024SpinQuant: LLM quantization with learned rotationscited by 471
  9. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 414
  10. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 367
  11. 2018Insights on representational similarity in neural networks with canonical correlationcited by 324
  12. 2019HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networkscited by 261
3 more
  1. 2020Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depthcited by 245
  2. 2024Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Modelscited by 146
  3. 2024Model merging with SVD to tie the Knotscited by 122

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026HeRo-Q: A General Framework for Stable Low Bit Quantization via Hessian Conditioningcited by 1
  2. 2026RAMP: Reinforcement Adaptive Mixed Precision Quantization for Efficient On Device LLM Inferencecited by 0
  3. 2026Predicting Quantization Price for Selecting PTQ Configurations Before Deploymentcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Layer-Wise Quantization: A Pragmatic and Effective Method for Quantizing LLMs Beyond Integer Bit-Levels
  2. 2024LSAQ: Layer-Specific Adaptive Quantization for Large Language Model Deployment
  3. 2025FlexQuant: A Flexible and Efficient Dynamic Precision Switching Framework for LLM Quantization
  4. 2026QTALE: Quantization-Robust Token-Adaptive Layer Execution for LLMs
  5. 2025Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantization
  6. 2023QuantEase: Optimization-based Quantization for Language Models
  7. 2025Layer-Wise High-Impact Parameter Ratio Optimization in Post-Training Quantization for Large Language Models
  8. 2025Adaptive Layer-Wise Transformations for Post-Training Quantization of Large Language Models