mlsanityData to 14 Oct 2026

Structured Transforms for Low-Overhead Quantization of Language Models

arXiv 2609.11687 ↗First posted 10 Sep 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

We revisit Kashin-decomposition-based weight quantization for large language models and propose an improved algorithm with stronger convergence properties and structured, efficient orthogonal transforms. The method retains the core factorization of each weight into two components -- one with bounded infinity norm and the other with bounded infinity norm after an orthogonal transformation -- but replaces the dense random orthogonal matrix with a sign-randomized Discrete Cosine Transform (DCT), reducing the per-iteration cost from O(N2) to O(N log N). The proposed greedy algorithm with alternating updates guarantees the four-peak distribution required for stable 2-bit clustering of each factor and admits closed-form initialization of cluster centers, removing the multi-restart k-means bottleneck of prior work. Composed with OPTQ-style sequential error compensation and QuIP-style incoherence preprocessing, the resulting JAX pipeline is competitive with OPTQ, QuIP, QuIP-RG and a fine-tuning- and vector-quantization-free variant of QuIP# at 4-bit per channel on OPT, Llama-2 and Pythia, with favorable wall-clock scaling. The bounded-ℓ_∞ factorization is also notably robust: on stress configurations where QuIP variants diverge to four-digit perplexity (Pythia-6.9B) or abort with NaNs in LDL back-substitution (Mistral-7B), Kashin-DCT remains numerically stable and stays close to FP16 baseline. At inference time, each weight decomposes into two 2-bit factor codes per channel that are structurally suited to native-2-bit hardware.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 23 of its 30 references that are in the map. First posted on arXiv 10 Sep 2026.

Builds on

23 of 30 references.

References · 23

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  2. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  3. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  4. 2016Pointer Sentinel Mixture Modelscited by 3,096
  5. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  6. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  7. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  8. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  9. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  10. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  11. 2021A White Paper on Neural Network Quantizationcited by 472
  12. 2024SpinQuant: LLM quantization with learned rotationscited by 471
11 more
  1. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 424
  2. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 414
  3. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 378
  4. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 322
  5. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 294
  6. 2024Extreme Compression of Large Language Models via Additive Quantizationcited by 232
  7. 2024DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMscited by 167
  8. 2024QTIP: Quantization with Trellises and Incoherence Processingcited by 91
  9. 2024GPTVQ: The Blessing of Dimensionality for LLM Quantizationcited by 82
  10. 2024MARLIN: Mixed-Precision Auto-Regressive Parallel Inference on Large Language Modelscited by 52
  11. 2024Quantization of Large Language Models with an Overdetermined Basiscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023QuantEase: Optimization-based Quantization for Language Models
  2. 2026TWLA: Achieving Ternary Weights and Low-Bit Activations for LLMs via Post-Training Quantization
  3. 2026BWLA: Breaking the Barrier of W1AX Post-Training Quantization for LLMs
  4. 2024DFRot: Achieving Outlier-Free and Massive Activation-Free for Rotated LLMs with Refined Rotation
  5. 2024Data-free Weight Compress and Denoise for Large Language Models
  6. 2025FPTQuant: Function-Preserving Transforms for LLM Quantization
  7. 2024LRQ: Optimizing Post-Training Quantization for Large Language Models by Learning Low-Rank Weight-Scaling Matrices
  8. 2025OstQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fitting