mlsanityData to 7 Oct 2026

SPHQuant: Efficient extreme low bit weight quantization for Vision-Language Models

arXiv 2609.24875 ↗First posted 21 Sep 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

Abstract (arXiv)

Recent foundation models are moving toward native multimodal Vision-Language Models (VLMs), making VLMs a central form of next-generation foundation models. However, their large language backbones make edge deployment difficult due to high memory footprint and memory-bound autoregressive decoding. Weight-only post-training quantization is a practical solution, but pushing VLMs to extreme low bit-widths remains challenging: existing rotation-free methods suffer from outliers at 2-3 bits, while rotation-based methods improve accuracy at the cost of additional runtime overhead. We propose SPHQuant, a rotation-free spherical weight-only quantization framework for VLMs. Instead of quantizing weights directly in Cartesian coordinates, SPHQuant decomposes each 8D weight vector into coordinate signs, radius, and a positive unit direction. This representation isolates outlier magnitude into the radius while keeping directions bounded and statistically regular. Based on this insight, SPHQuant allocates extra precision to the radius to mitigate accuracy degradation induced by outliers. It further uses a compact positive-direction codebook and fine-tunes codebook entries through angular parameterization to preserve the unit-sphere constraint. We also design a hardware-friendly GEMV kernel that keeps the direction codebook small enough for shared-memory lookup and packs radial bits efficiently. Experiments show that SPHQuant matches the performance of state-of-the-art extreme low-bit quantization methods while improving decode throughput over QTIP by 30.3% on RTX A6000. Code will be released in https://github.com/Pushazf/SPHQuant.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 18 of its 31 references that are in the map. First posted on arXiv 21 Sep 2026.

Builds on

18 of 31 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026VEQ: Modality-Adaptive Quantization for MoE Vision-Language Models1 Feb

Other references · 17

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  3. 2019Towards VQA Models That Can Readcited by 1,696
  4. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  5. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 893
  6. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 494
  7. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 397
  8. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 354
  9. 2024MobileVLM V2: Faster and Stronger Baseline for Vision Language Modelcited by 172
  10. 2024Efficient Multimodal Large Language Models: A Surveycited by 95
  11. 2024QTIP: Quantization with Trellises and Incoherence Processingcited by 86
  12. 2025ParoQuant: Pairwise Rotation Quantization for Efficient Reasoning LLM Inferencecited by 16
5 more
  1. 2025PCDVQ: Enhancing Vector Quantization for Large Language Models via Polar Coordinate Decouplingcited by 5
  2. 2024Pyramid Vector Quantization for LLMscited by 4
  3. 2025QuantVSR: Low-Bit Post-Training Quantization for Real-World Video Super-Resolutioncited by 2
  4. 2025QuantDemoire: Quantization with Outlier Aware for Image Demoir'eingcited by 1
  5. 2026LSGQuant: Layer-Sensitivity Guided Quantization for One-Step Diffusion Real-World Video Super-Resolutioncited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Bi-VLM: Pushing Ultra-Low Precision Post-Training Quantization Boundaries in Vision-Language Models
  2. 2026Beyond Reconstruction Loss in Post-Training Quantization: Balanced Fitting for Large Vision-Language Models
  3. 2024Q-VLM: Post-training Quantization for Large Vision-Language Models
  4. 2026Towards Understanding Best Practices for Quantization of Vision-Language Models
  5. 2026RGSQ: Riemannian Geometry-Sensitive Quantization for Large Vision-Language Models
  6. 2025QSVD: Efficient Low-rank Approximation for Unified Query-Key-Value Weight Compression in Low-Precision Vision-Language Models
  7. 2026Breaking Modality Heterogeneity in Low-Bit Quantization for Large Vision-Language Models
  8. 2026SAB-LVLM: Significance-Aware Binarization for Large Vision-Language Models