mlsanityData to 14 Oct 2026

HyperQuant: A Rate-Distortion-Optimal Quantization Pipeline for Large Language and Diffusion Models

arXiv 2606.23406 ↗First posted 22 Jun 2026cs.LG cs.AIcited by 2

On the map: Diffusion models → Diffusion foundations and sampling

also in 1 more
Abstract (arXiv)

We present HyperQuant (Hadamard, optimallY Packing, Entropy Rice-coding), a unified post-training quantization pipeline for the weights and the KV cache of large language and diffusion transformers. Across a suite of self-contained experiments (Table 1), HyperQuant outperforms the recent HIGGS scheme at every operating point from 3 to 5 bits per scalar (bps) on weights, and beats both TurboQuant and OCTOPUS on KV quantization down to 1.7 bps. Beyond the LLM setting, HyperQuant quantizes the 19B-parameter LTX-2 DiT video model with no observable per-frame artifacts. End-to-end on an H100 at 4 bps, HyperQuant compresses the linear weights ~3.9x and the KV cache ~3.79x at near-lossless quality. HyperQuant combines four known ideas into a single construction: (i) a per-tile Randomized Hadamard Transform that makes the per-coordinate distribution of weights and activations approximately Gaussian; (ii) quantization to a low-dimensional optimal lattice (E8, D4, A2, or Z); (iii) lossless bit-stripping and near-entropy-optimal variable-length Rice coding of the lattice indices; and (iv) bias-correction methods for the KV cache that keep the reconstruction unbiased under inner products, preserving attention semantics. We further integrate the pipeline with 8-bit and 4-bit Tensor-Core MMA paths (fp8-e4m3, int8, nvfp4, mxfp4), and find that int8 beats fp8 on the post-RHT lattice output. Project page: https://moonmath.ai/hyperquant/

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 19 of its 43 references that are in the map. First posted on arXiv 22 Jun 2026.

Builds on

19 of 43 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesis5 Mar
  2. 2022Scalable Diffusion Models with Transformers19 Dec

Other references · 17

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2016Pointer Sentinel Mixture Modelscited by 3,096
  3. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  4. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  5. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  6. 2022Efficiently Scaling Transformer Inferencecited by 539
  7. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 424
  8. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 414
  9. 2024SliceGPT: Compress Large Language Models by Deleting Rows and Columnscited by 385
  10. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 378
  11. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 367
  12. 2022FP8 Formats for Deep Learningcited by 196
5 more
  1. 2023Microscaling Data Formats for Deep Learningcited by 152
  2. 2024QJL: 1-Bit Quantized JL Transform for KV Cache Quantization with Zero Overheadcited by 52
  3. 2025NestQuant: Nested Lattice Quantization for Matrix Products and LLMscited by 23
  4. 2024Pushing the Limits of Large Language Model Quantization via the Linearity Theoremcited by 21
  5. 2026OCTOPUS: Optimized KV Cache for Transformers via Octahedral Parametrization Under optimal Squared error quantizationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026EntroPack: Fast and Accurate Entropy-Coded Weight Compression at Arbitrary Bitratescited by 0
  2. 2026CORAL-KV: Compact Cross-Layer Prediction and Residual Coding for Autoregressive Video KV Cachescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantization
  2. 2026PolarQuant: Optimal Gaussian Weight Quantization via Hadamard Rotation for LLM Compression
  3. 2024WKVQuant: Quantizing Weight and Key/Value Cache for Large Language Models Gains More
  4. 2024CrossQuant: A Post-Training Quantization Method with Smaller Quantization Kernel for Precise Large Language Model Compression
  5. 2025Universality of Layer-Level Entropy-Weighted Quantization Beyond Model Architecture and Size
  6. 2024CDQuant: Greedy Coordinate Descent for Accurate LLM Quantization
  7. 2026On the Quantization Robustness of Diffusion Language Models in Coding Benchmarks
  8. 2024FlattenQuant: Breaking Through the Inference Compute-bound for Large Language Models with Per-tensor Quantization