mlsanityData to 7 Oct 2026

Breaking Modality Heterogeneity in Low-Bit Quantization for Large Vision-Language Models

arXiv 2605.19929 ↗First posted 19 May 2026cs.CV cs.AIcited by 1

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

Abstract (arXiv)

Low-bit post-training quantization (PTQ) is a pivotal technique for deploying Vision-Language Models (VLMs) on resource-constrained devices. However, existing PTQ methods often degrade VLMs' accuracy due to the heterogeneous activation distributions of text and vision modalities during quantization. We find that this cross-modal heterogeneity is distributed unevenly across channels: a small subset of channels contains most modality-specific outliers, and these outliers typically reside in different channels for each modality. Motivated by this, we propose SplitQ, a channel-Splitting-driven post-training Quantization framework. At its core, SplitQ introduces a novel Modality-specific Outlier Channel Decoupling (MOCD) module that effectively isolates salient modality-specific outlier channels with minimal overhead. To further address the remaining cross-modal distribution discrepancies, we design an Adaptive Cross-Modal Calibration (ACC) module that employs dual lightweight learnable branches to dynamically mitigate modality-induced quantization errors. Extensive experiments on popular VLMs demonstrate that SplitQ significantly outperforms existing approaches across 6 popular multi-modal datasets under all evaluated quantization settings, including W4A8, W4A4, W3A3, and W3A2. Notably, SplitQ preserves 93.5% of FP16 performance under the challenging W3A3 setting (69.5 vs. 74.3), pushing the efficiency frontier for deploying advanced VLMs. Our code is available at https://github.com/EMVision-NK/SplitQ

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 27 of its 61 references that are in the map. First posted on arXiv 19 May 2026.

Builds on

27 of 61 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026WSVD: Weighted Low-Rank Approximation for Fast and Efficient Execution of Low-Precision Vision-Language Models2 Apr
  2. 2025QSVD: Efficient Low-rank Approximation for Unified Query-Key-Value Weight Compression in Low-Precision Vision-Language Models18 Oct
  3. 2025SalQ-VLM: Fine-Grained Saliency-Guided Quantization for Vision-Language Models5 Aug

Other references · 24

  1. 2023Qwen Technical Reportcited by 3,273
  2. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  3. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 929
  4. 2024SpinQuant: LLM quantization with learned rotationscited by 446
  5. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 403
  6. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 359
  7. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 285
  8. 2024OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoningcited by 165
  9. 2024SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Modelscited by 161
  10. 2024FlatQuant: Flatness Matters for LLM Quantizationcited by 116
  11. 2025OstQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fittingcited by 94
  12. 2023ZeroQuant-V2: Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensationcited by 78
12 more
  1. 2024AffineQuant: Affine Transformation Quantization for Large Language Modelscited by 64
  2. 2024Low-Rank Quantization-Aware Training for LLMscited by 42
  3. 2023CBQ: Cross-Block Quantization for Large Language Modelscited by 34
  4. 2025MoEQuant: Enhancing Quantization for Mixture-of-Experts Large Language Models via Expert-Balanced Sampling and Affinity Guidancecited by 32
  5. 2024I-LLM: Efficient Integer-Only Inference for Fully-Quantized Low-Bit Large Language Modelscited by 19
  6. 2024MixLLM: LLM Quantization with Global Mixed-precision between Output-features and Highly-efficient System Designcited by 17
  7. 2023FPTQ: Fine-grained Post-Training Quantization for Large Language Modelscited by 16
  8. 2024QQQ: Quality Quattuor-Bit Quantization for Large Language Modelscited by 14
  9. 2024Post Training Quantization of Large Language Models with Microscaling Formatscited by 10
  10. 2026MASQuant: Modality-Aware Smoothing Quantization for Multimodal Large Language Modelscited by 5
  11. 2026SliderQuant: Accurate Post-Training Quantization for LLMscited by 5
  12. 2025RWKVQuant: Quantizing the RWKV Family with Proxy Guided Hybrid of Scalar and Vector Quantizationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Beyond Independent Optimization: Compression, MoE Routing, and Quantization Interactions in Multimodal Edge Intelligencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Fine-Grained Post-Training Quantization for Large Vision Language Models with Quantization-Aware Integrated Gradients
  2. 2026VEQ: Modality-Adaptive Quantization for MoE Vision-Language Models
  3. 2025Bi-VLM: Pushing Ultra-Low Precision Post-Training Quantization Boundaries in Vision-Language Models
  4. 2024Q-VLM: Post-training Quantization for Large Vision-Language Models
  5. 2024MBQ: Modality-Balanced Quantization for Large Vision-Language Models
  6. 2026Beyond Reconstruction Loss in Post-Training Quantization: Balanced Fitting for Large Vision-Language Models
  7. 2024Deciphering Cross-Modal Alignment in Large Vision-Language Models with Modality Integration Rate
  8. 2024Split to Merge: Unifying Separated Modalities for Unsupervised Domain Adaptation