mlsanityData to 14 Oct 2026

MorphoQuant: Modality-Aware Quantization for Omni-modal Large Language Models

arXiv 2606.04349 ↗First posted 3 Jun 2026cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Conventional Post-Training Quantization (PTQ) methods struggle with 4-bit Omni-modal Large Language Models (OLLMs) due to the extreme distribution heterogeneity and disparate outlier patterns across modalities. To address this, we propose MorphoQuant, a modality-aware PTQ framework engineered to preserve cross-modal morphology and mitigate outlier loss. Specifically, we introduce Distribution-Aware Bias Compensation (DABC), which selectively absorbs long-tailed outliers into channel-wise biases. This mechanism safeguards outlier magnitudes while maintaining high-precision discretization for dense inliers, thereby preserving accurate discretization across diverse modal distribution. Complementing this, we propose Morphology-Directed Quantization Function Optimization (MDQFO) to co-optimize the quantization grid with the bias mask, ensuring fine-grained alignment across modalities. Extensive evaluations on Qwen2.5-Omni across benchmarks like MMMU and Video-MME demonstrate our approach's superiority. Notably, our W4A4 model achieves 76.63% on ScienceQA, significantly outperforming SOTA W4A4 methods and surprisingly surpassing the W4A16 baseline, which fully demonstrates the exceptional accuracy-efficiency trade-off of our framework.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 33 of its 36 references that are in the map. First posted on arXiv 3 Jun 2026.

Builds on

33 of 36 references.

References · 33

  1. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  2. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  3. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  4. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  5. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 526
  6. 2025Qwen3-Omni Technical Reportcited by 497
  7. 2024SpinQuant: LLM quantization with learned rotationscited by 471
  8. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 424
  9. 2022Multimodal Learning with Transformers: A Surveycited by 400
  10. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 367
  11. 2021LAVT: Language-Aware Vision Transformer for Referring Image Segmentationcited by 348
  12. 2023Atom: Low-bit Quantization for Efficient and Accurate LLM Servingcited by 272
21 more
  1. 2023LAMM: Language-Assisted Multi-Modal Instruction-Tuning Dataset, Framework, and Benchmarkcited by 169
  2. 2024BiLLM: Pushing the Limit of Post-Training Quantization for LLMscited by 144
  3. 2024The Revolution of Multimodal Large Language Models: A Surveycited by 137
  4. 2024Fit and Prune: Fast and Training-free Visual Token Pruning for Multi-modal Large Language Modelscited by 126
  5. 2024Efficient Multimodal Large Language Models: A Surveycited by 96
  6. 2024LLaVA-MoLE: Sparse Mixture of LoRA Experts for Mitigating Data Conflicts in Instruction Finetuning MLLMscited by 84
  7. 2024A Survey on Benchmarks of Multimodal Large Language Modelscited by 82
  8. 2024Datasets for Large Language Models: A Comprehensive Surveycited by 71
  9. 2024Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directionscited by 68
  10. 2024Beyond Efficiency: A Systematic Survey of Resource-Efficient Large Language Modelscited by 67
  11. 2024An empirical study of LLaMA3 quantization: from LLMs to MLLMscited by 64
  12. 2023Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scalingcited by 61
  13. 2020EasyQuant: Post-training Quantization via Scale Optimizationcited by 57
  14. 2023Towards Accurate Post-training Quantization for Diffusion Modelscited by 47
  15. 2024Q-VLM: Post-training Quantization for Large Vision-Language Modelscited by 47
  16. 2024MBQ: Modality-Balanced Quantization for Large Vision-Language Modelscited by 45
  17. 2025OmniVTLA: Vision-Tactile-Language-Action Models with Semantic-Aligned Tactile Sensingcited by 38
  18. 2025Multi-agent Embodied AI: Advances and Future Directionscited by 36
  19. 2025MQuant: Unleashing the Inference Potential of Multimodal Large Language Models via Full Static Quantizationcited by 35
  20. 2025A Survey on Efficient Vision-Language Modelscited by 19
  21. 2024Advancing Multimodal Large Language Models with Quantization-Aware Scale Learning for Efficient Adaptationcited by 15

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026MASQuant: Modality-Aware Smoothing Quantization for Multimodal Large Language Models
  2. 2026Breaking Modality Heterogeneity in Low-Bit Quantization for Large Vision-Language Models
  3. 2026MosaicQuant: Inlier-Outlier Disaggregation for Unified 4-Bit LLM Quantization
  4. 2024OutlierTune: Efficient Channel-Wise Quantization for Large Language Models
  5. 2024Post Training Quantization of Large Language Models with Microscaling Formats
  6. 2026RDQ: Residual Distribution Quantization for Large Language Models
  7. 2024LeanQuant: Accurate and Scalable Large Language Model Quantization with Loss-error-aware Grid
  8. 2024DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMs