mlsanityData to 14 Oct 2026

PrismQuant: Optimal Null-Space Rotations for Grouped Quantizers

arXiv 2609.32429 ↗First posted 26 Sep 2026cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Smaller activation outliers do not necessarily imply better low-bit quantization: their alignment with the quantizer matters. We introduce PrismQuant, a quantizer-aware rotation framework that aligns the leading activation eigenspace with the constant group subspace of asymmetric grouped INT4. The affine offsets represent the energy in this subspace without widening the range within the group. We formulate rotation design as a Ky Fan trace maximization and derive a closed-form solution that is provably optimal for this alignment objective. Compact Householder transformations and their compact-WY representation enable gradient-free construction and efficient application at both foldable and online sites. A predictive range law further connects unaligned activation energy and group size to quantization-relevant variation. Experiments on Llama, Qwen, and Mistral span dense models up to 70B parameters and a 30B mixture-of-experts model. Under W4A4KV4, PrismQuant sets the state of the art on Llama-3.2-3B among the compared methods in both perplexity and accuracy. On Llama-3.1-70B, it attains 3.85 perplexity and 72.46% average zero-shot accuracy, only 0.22 percentage points below full precision. In the deployment study on Llama-3.1-8B, our optimized implementation achieves 1.51x prefill and 1.22x CUDA Graph decode speedups over matched FP16 baselines, with 56.34% lower decode peak memory and only 2.35% additional Graph decode latency over Hadamard. Code and models are available at https://github.com/ForeverBlue816/PrismQuant and https://huggingface.co/collections/ForeverBlue/prsimquant.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 14 of its 39 references that are in the map. First posted on arXiv 26 Sep 2026.

Builds on

14 of 39 references.

References · 14

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 637
  5. 2024Massive Activations in Large Language Modelscited by 294
  6. 2024FlatQuant: Flatness Matters for LLM Quantizationcited by 131
  7. 2023RPTQ: Reorder-based Post-training Quantization for Large Language Modelscited by 116
  8. 2025OstQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fittingcited by 97
  9. 2024ResQ: Mixed-Precision Quantization of Large Language Models with Low-Rank Residualscited by 36
  10. 2024DFRot: Achieving Outlier-Free and Massive Activation-Free for Rotated LLMs with Refined Rotationcited by 17
  11. 2026Gated Relational Alignment via Confidence-based Distillation for Efficient VLMscited by 6
  12. 2025BASE-Q: Bias and Asymmetric Scaling Enhanced Rotational Quantization for Large Language Modelscited by 5
2 more
  1. 2025Demystifying Singular Defects in Large Language Modelscited by 2
  2. 2026OffQ: Taming Structured Outliers in LLM Quantization by Offsettingcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026ThinQuant: Scalable Rotation Learning for Weight and Activation Quantization of LLMs
  2. 2026GyRot: Leveraging Hidden Synergy between Rotation and Fine-grained Group Quantization for Low-bit LLM Inference
  3. 2025Grouped Sequency-arranged Rotation: Optimizing Rotation Transformation for Quantization for Free
  4. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs
  5. 2026ConQuR: Corner Aligned Activation Quantization via Optimized Rotations for LLMs
  6. 2025Outlier Smoothing with Closed-Form Rotations for W4A4 Large Language Model Quantization
  7. 2026DuQuant++: Fine-grained Rotation Enhances Microscaling FP4 Quantization
  8. 2025DartQuant: Efficient Rotational Distribution Calibration for LLM Quantization