mlsanityData to 14 Oct 2026

DAQ: Delta-Aware Quantization for Post-Training LLM Weight Compression

arXiv 2603.22324 ↗First posted 20 Mar 2026cs.LG cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

We introduce Delta-Aware Quantization (DAQ), a data-free post-training quantization framework that preserves the knowledge acquired during post-training. Standard quantization objectives minimize reconstruction error but are agnostic to the base model, allowing quantization noise to disproportionately corrupt the small-magnitude parameter deltas (Delta W) that encode post-training behavior -- an effect we analyze through the lens of quantization as implicit regularization. DAQ replaces reconstruction-based objectives with two delta-aware metrics -- Sign Preservation Rate and Cosine Similarity -- that directly optimize for directional fidelity of Delta W, requiring only the base and post-trained weight matrices. In a pilot FP8 study, DAQ recovers style-specific capabilities lost under standard quantization while maintaining general performance.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 17 of its 20 references that are in the map. First posted on arXiv 20 Mar 2026.

Builds on

17 of 20 references.

References · 17

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  5. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  6. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  7. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  8. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  9. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
  10. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 592
  11. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 526
  12. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 424
5 more
  1. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 367
  2. 2022FP8 Formats for Deep Learningcited by 196
  3. 2023ZeroQuant-V2: Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensationcited by 79
  4. 2023Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMscited by 50
  5. 2026AngelSlim: A more accessible, comprehensive, and efficient toolkit for large model compressioncited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Lost in the bf16 Cast: Exporting Ternary Language Models Can Revert Most Low-Learning-Rate Code Changescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026When Lower Reconstruction Loss Hurts: Distributionally Robust Refinement for Low-Bit LLM Quantization
  2. 2026The Devil Is in the Reconstruction Loss Scale: Rethinking Optimization in LLM Quantization
  3. 2024DeltaDQ: Ultra-High Delta Compression for Fine-Tuned LLMs via Group-wise Dropout and Separate Quantization
  4. 2026D2Quant: Accurate Low-bit Post-Training Weight Quantization for LLMs
  5. 2025SignRoundV2: Toward Closing the Performance Gap in Extremely Low-Bit Post-Training Quantization for LLMs
  6. 2026Few Bits, One Law: Toward W2A4KV2
  7. 2025Benchmarking Post-Training Quantization in LLMs: Comprehensive Taxonomy, Unified Evaluation, and Comparative Analysis
  8. 2026ProjQ: Project-and-Quantize for Adapter-Aware LLM Compression