mlsanityData to 14 Oct 2026

SDQ: Sparse Decomposed Quantization for LLM Inference

arXiv 2406.13868 ↗First posted 19 Jun 2024cs.LG cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Recently, large language models (LLMs) have shown surprising performance in task-specific workloads as well as general tasks with the given prompts. However, to achieve unprecedented performance, recent LLMs use billions to trillions of parameters, which hinder the wide adaptation of those models due to their extremely large compute and memory requirements. To resolve the issue, various model compression methods are being actively investigated. In this work, we propose SDQ (Sparse Decomposed Quantization) to exploit both structured sparsity and quantization to achieve both high compute and memory efficiency. From our evaluations, we observe that SDQ can achieve 4x effective compute throughput with <1% quality drop.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 18 of its 32 references that are in the map. First posted on arXiv 19 Jun 2024.

Builds on

18 of 32 references.

References · 18

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  4. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  5. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  6. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  7. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 800
  8. 2021Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networkscited by 527
  9. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 322
  10. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 294
  11. 2019MLPerf Inference Benchmarkcited by 199
  12. 2023Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsitycited by 165
6 more
  1. 2023OliVe: Accelerating Large Language Models via Hardware-friendly Outlier-Victim Pair Quantizationcited by 92
  2. 2021VS-Quant: Per-vector Scaled Quantization for Accurate Low-Precision Neural Network Inferencecited by 64
  3. 2023With Shared Microexponents, A Little Shifting Goes a Long Waycited by 50
  4. 2021S2TA: Exploiting Structured Sparsity for Energy-Efficient Mobile CNN Accelerationcited by 19
  5. 2022Sparseloop: An Analytical Approach To Sparse Tensor Accelerator Modelingcited by 15
  6. 2023HighLight: Efficient and Flexible DNN Acceleration with Hierarchical Structured Sparsitycited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 1.

4 papers.

  1. 2024MicroScopiQ: Accelerating Foundational Models through Outlier-Aware Microscaling Quantizationcited by 28
  2. 2025Semantic Retention and Extreme Compression in LLMs: Can We Have Both?cited by 2
  3. 2025Amber Pruner: Leveraging N:M Activation Sparsity for Efficient Prefill in Large Language Modelscited by 1
  4. 2026APT: Accelerating Diffusion Transformers via Attention Probability-Guided Pruning and Quantizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025SDQ-LLM: Sigma-Delta Quantization for 1-bit LLMs of any size
  2. 2024ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models
  3. 2026HAS-VQ: Hessian-Adaptive Sparse Vector Quantization for High-Fidelity LLM Compression
  4. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression
  5. 2026Fitting Is Not Enough: Smoothness in Extremely Quantized LLMs
  6. 2025Accelerating LLM Inference with Flexible N:M Sparsity via A Fully Digital Compute-in-Memory Accelerator
  7. 2024GQSA: Group Quantization and Sparsity for Accelerating Large Language Model Inference
  8. 2024Q-Sparse: All Large Language Models can be Fully Sparsely-Activated