mlsanityData to 7 Oct 2026

QTIP: Quantization with Trellises and Incoherence Processing

arXiv 2406.11235 ↗First posted 17 Jun 2024cs.LGcited by 86

Not on the map. Why?

Abstract (arXiv)

Post-training quantization (PTQ) reduces the memory footprint of LLMs by quantizing weights to low-precision datatypes. Since LLM inference is usually memory-bound, PTQ methods can improve inference throughput. Recent state-of-the-art PTQ approaches use vector quantization (VQ) to quantize multiple weights at once, which improves information utilization through better shaping. However, VQ requires a codebook with size exponential in the dimension. This limits current VQ-based PTQ works to low VQ dimensions (≤ 8) that in turn limit quantization quality. Here, we introduce QTIP, which instead uses trellis coded quantization (TCQ) to achieve ultra-high-dimensional quantization. TCQ uses a stateful decoder that separates the codebook size from the bitrate and effective dimension. QTIP introduces a spectrum of lookup-only to computed lookup-free trellis codes designed for a hardware-efficient "bitshift" trellis structure; these codes achieve state-of-the-art results in both quantization quality and inference speed.

Key facts. Cited by 86 papers in the map (data through 7 Oct 2026). Builds on 10 of its 36 references that are in the map. First posted on arXiv 17 Jun 2024.

Builds on

10 of 36 references.

References · 10

  1. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 821
  2. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
  3. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 598
  4. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 403
  5. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 397
  6. 2022The case for 4-bit precision: k-bit Inference Scaling Lawscited by 247
  7. 2024Extreme Compression of Large Language Models via Additive Quantizationcited by 223
  8. 2023HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolutioncited by 187
  9. 2024GPTVQ: The Blessing of Dimensionality for LLM Quantizationcited by 78
  10. 2024PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compressioncited by 59

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 38; 2026: 44.

86 papers.

Sort:

By area:

  1. 2024A Survey of Low-bit Large Language Models: Basics, Systems, and Algorithmscited by 30
  2. 2025Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantizationcited by 30
  3. 2025GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidancecited by 28
  4. 2025Qronos: Correcting the Past by Shaping the Future... in Post-Training Quantizationcited by 24
  5. 2025Cache Me If You Must: Adaptive Key-Value Quantization for Large Language Modelscited by 21
  6. 2025Model-Preserving Adaptive Roundingcited by 20
  7. 2025Better Tokens for Better 3D: Advancing Vision-Language Modeling in 3D Medical Imagingcited by 20
  8. 2025The Geometry of LLM Quantization: GPTQ as Babai's Nearest Plane Algorithmcited by 18
  9. 2024MixLLM: LLM Quantization with Global Mixed-precision between Output-features and Highly-efficient System Designcited by 17
  10. 2025NestQuant: Nested Lattice Quantization for Matrix Products and LLMscited by 16
  11. 2025ParoQuant: Pairwise Rotation Quantization for Efficient Reasoning LLM Inferencecited by 16
  12. 2025WUSH: Near-Optimal Adaptive Transforms for LLM Quantizationcited by 11
38 more
  1. 2024Optimal Quantization for Matrix Multiplicationcited by 10
  2. 2025Identifying Sensitive Weights via Post-quantization Integralcited by 10
  3. 2025ICQuant: Index Coding enables Low-bit LLM Quantizationcited by 9
  4. 2025PTQTP: Post-Training Quantization to Trit-Planes for Large Language Modelscited by 9
  5. 2025Q-Palette: Fractional-Bit Quantizers Toward Optimal Bit Allocation for Efficient LLM Deploymentcited by 9
  6. 2026DuQuant++: Fine-grained Rotation Enhances Microscaling FP4 Quantizationcited by 8
  7. 2026ActQuant: Sub-4-bit Action-Guided Quantization for Vision-Language-Action Modelscited by 7
  8. 2025DenoiseRotator: Enhance Pruning Robustness for LLMs via Importance Concentrationcited by 6
  9. 2025MACKO: Sparse Matrix-Vector Multiplication for Low Sparsitycited by 6
  10. 2026Leech Lattice Vector Quantization for Efficient LLM Compressioncited by 6
  11. 2025PCDVQ: Enhancing Vector Quantization for Large Language Models via Polar Coordinate Decouplingcited by 5
  12. 2025any4: Learned 4-bit Numeric Representation for LLMscited by 5
  13. 2026WaterSIC: Information-Theoretically (Near) Optimal Linear Layer Quantizationcited by 5
  14. 2025Learning Grouped Lattice Vector Quantizers for Low-Bit LLM Compressioncited by 4
  15. 2025LUT-LLM: Efficient Large Language Model Inference with Memory-based Computations on FPGAscited by 4
  16. 2026GSQ: Highly-Accurate Low-Precision Scalar Quantization for LLMs via Gumbel-Softmax Samplingcited by 4
  17. 2026Statistically-Lossless Quantization of Large Language Modelscited by 4
  18. 2026Quantizing With Randomized Hadamard Transforms: Efficient Heuristic Now Provencited by 4
  19. 2026HARP: Hadamard-Preconditioned Adaptive Rotation Processor for Extreme LLM Quantizationcited by 4
  20. 2025RSQ: Learning from Important Tokens Leads to Better Quantized LLMscited by 3
  21. 2025Addition is almost all you need: Compressing large language models with double binary factorizationcited by 3
  22. 2025CCQ: Convolutional Code for Extreme Low-bit Quantization in LLMscited by 3
  23. 2025Exploring Layer-wise Information Effectiveness for Post-Training Quantization in Small Language Modelscited by 3
  24. 2025PocketLLM: Ultimate Compression of Large Language Models via Meta Networkscited by 3
  25. 2025CodeGEMM: A Codebook-Centric Approach to Efficient GEMM in Quantized LLMscited by 3
  26. 2025More Than Bits: Multi-Envelope Double Binary Factorization for Extreme Quantizationcited by 3
  27. 2026ScaleBITS: Scalable Bitwidth Search for Hardware-Aligned Mixed-Precision LLMscited by 3
  28. 2025DiscQuant: A Quantization Method for Neural Networks Inspired by Discrepancy Theorycited by 2
  29. 2025Highly Efficient and Effective LLMs with Multi-Boolean Architecturescited by 2
  30. 2025PATCH: Learnable Tile-level Hybrid Sparsity for LLMscited by 2
  31. 2025RSAVQ: Riemannian Sensitivity-Aware Vector Quantization for Large Language Modelscited by 2
  32. 2026EdgeRazor: A Lightweight Framework for Large Language Models via Mixed-Precision Quantization-Aware Distillationcited by 2
  33. 2026Grid Games: The Power of Multiple Grids for Quantizing Large Language Modelscited by 2
  34. 2026High-Rate Quantized Matrix Multiplication IIcited by 2
  35. 2026LiftQuant: Continuous Bit-Width LLM via Dimensional Lifting and Projectioncited by 2
  36. 2026LC-QAT: Data-Efficient 2-Bit QAT for LLMs via Linear-Constrained Vector Quantizationcited by 2
  37. 2026SoftWater: Class-Aware Rate Allocation for Softmax Quantizationcited by 2
  38. 2024Scaling Laws For Mixed Quantizationcited by 1

50 most cited of 86 All 86 (JSON)