mlsanityData to 7 Oct 2026

QUICK: Quantization-aware Interleaving and Conflict-free Kernel for efficient LLM inference

arXiv 2402.10076 ↗First posted 15 Feb 2024cs.LG cs.AI cs.CLcited by 6

On the map: Efficient LLM inference and serving → KV cache quantization

Abstract (arXiv)

We introduce QUICK, a group of novel optimized CUDA kernels for the efficient inference of quantized Large Language Models (LLMs). QUICK addresses the shared memory bank-conflict problem of state-of-the-art mixed precision matrix multiplication kernels. Our method interleaves the quantized weight matrices of LLMs offline to skip the shared memory write-back after the dequantization. We demonstrate up to 1.91x speedup over existing kernels of AutoAWQ on larger batches and up to 1.94x throughput gain on representative LLM models on various NVIDIA GPU devices.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in KV cache quantization, one of 12 directions in Efficient LLM inference and serving. Builds on 12 of its 15 references that are in the map. First posted on arXiv 15 Feb 2024.

Builds on

12 of 15 references.

References · 12

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  4. 2023Mistral 7Bcited by 3,065
  5. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  6. 2024Mixtral of Expertscited by 1,785
  7. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  8. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  9. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,090
  10. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  11. 2024SliceGPT: Compress Large Language Models by Deleting Rows and Columnscited by 378
  12. 2023OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Modelscited by 113

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 3; 2026: 2.

6 papers.

  1. 2024Mixture of Scales: Memory-Efficient Token-Adaptive Binarization for Large Language Modelscited by 14
  2. 2025LMDeploy Accelerates Mixed-Precision LLM Inference with TurboMindcited by 10
  3. 2025ROSAQ: Rotation-based Saliency-Aware Weight Quantization for Efficiently Compressing Large Language Modelscited by 2
  4. 2025Q-resafe: Assessing Safety Risks and Quantization-aware Safety Patching for Quantized Large Language Modelscited by 2
  5. 2026A Comparative Study of CNN Optimization Methods for Edge AI: Exploring the Role of Early Exitscited by 0
  6. 2026StreamDQ: Near-Memory Weight DeQuantization in Custom HBM for Scalable AI Inference Accelerationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024MixPE: Quantization and Hardware Co-design for Efficient LLM Inference
  2. 2025VecInfer: Efficient LLM Inference with Low-Bit KV Cache via Outlier-Suppressed Vector Quantization
  3. 2023Fast and Efficient 2-bit LLM Inference on GPU: 2/4/16-bit in a Weight Matrix with Asynchronous Dequantization
  4. 2026GPU-Accelerated INT8 Quantization for KV Cache Compression in Large Language Models
  5. 2023Efficient LLM Inference on CPUs
  6. 2023Efficient LLM inference solution on Intel GPU
  7. 2024ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models
  8. 2024Efficient Arbitrary Precision Acceleration for Large Language Models on GPU Tensor Cores