mlsanityData to 14 Oct 2026

MQuant: Unleashing the Inference Potential of Multimodal Large Language Models via Full Static Quantization

arXiv 2502.00425 ↗First posted 1 Feb 2025cs.CV cs.AIcited by 35

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

Abstract (arXiv)

Multimodal large language models (MLLMs) have garnered widespread attention due to their ability to understand multimodal input. However, their large parameter sizes and substantial computational demands severely hinder their practical deployment and application.While quantization is an effective way to reduce model size and inference latency, its application to MLLMs remains underexplored. In this paper, we propose MQuant, a post-training quantization (PTQ) framework designed to tackle the unique challenges of multimodal large language models (MLLMs). Conventional quantization often struggles with MLLMs because of (a) high inference latency from large visual token counts, (b) distributional disparities between visual and textual tokens, and (c) extreme outliers introduced by Hadamard-based transformations. To address these issues, MQuant introduces: Modality-Specific Static Quantization (MSQ), assigning distinct static scales for visual vs. textual tokens; Attention-Invariant Flexible Switching (AIFS), reordering tokens to preserve casual attention while eliminating expensive token-wise scale computations; Rotation Magnitude Suppression (RMS), mitigating weight outliers arising from online Hadamard rotations. On five mainstream MLLMs (including Qwen-VL, MiniCPM-V, CogVLM2), MQuant under W4A8 achieves near-floating-point accuracy (<1% degradation) while reducing inference latency by up to 30%, significantly outperforming existing PTQ baselines. Our MQuant effectively bridges the gap for efficient and accurate MLLMs inference in resource-constrained devices. Code has been released in https://github.com/StiphyJay/MQuant.

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 38 of its 71 references that are in the map. First posted on arXiv 1 Feb 2025.

Builds on

38 of 71 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024MBQ: Modality-Balanced Quantization for Large Vision-Language Models27 Dec

Other references · 37

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2016Layer Normalizationcited by 5,669
  6. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  7. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  8. 2023Qwen Technical Reportcited by 3,344
  9. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  10. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  11. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  12. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
25 more
  1. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  2. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  3. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  4. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
  5. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
  6. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
  7. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 723
  8. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 708
  9. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  10. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 476
  11. 2024InternLM2 Technical Reportcited by 397
  12. 2024Mini-Gemini: Mining the Potential of Multi-modality Vision Language Modelscited by 365
  13. 2024CogVLM2: Visual Language Models for Image and Video Understandingcited by 217
  14. 2024MobileVLM V2: Faster and Stronger Baseline for Vision Language Modelcited by 174
  15. 2023ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Modelscited by 169
  16. 2024LLM Inference Unveiled: Survey and Roofline Model Insightscited by 152
  17. 2023RPTQ: Reorder-based Post-training Quantization for Large Language Modelscited by 116
  18. 2025OstQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fittingcited by 97
  19. 2024WKVQuant: Quantizing Weight and Key/Value Cache for Large Language Models Gains Morecited by 69
  20. 2024I-LLM: Efficient Integer-Only Inference for Fully-Quantized Low-Bit Large Language Modelscited by 23
  21. 2024MobileQuant: Mobile-friendly Quantization for On-device Language Modelscited by 17
  22. 2023FPTQ: Fine-grained Post-Training Quantization for Large Language Modelscited by 16
  23. 2024QQQ: Quality Quattuor-Bit Quantization for Large Language Modelscited by 16
  24. 2024PrefixQuant: Eliminating Outliers by Prefixed Tokens for Large Language Models Quantizationcited by 12
  25. 2024PTQ4RIS: Post-Training Quantization for Referring Image Segmentationcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 11; 2026: 24.

35 papers.

Sort:

By area:

  1. 2026ConeSep: Cone-based Robust Noise-Unlearning Compositional Network for Composed Image Retrievalcited by 17
  2. 2026Task-Specific Efficiency Analysis: When Small Language Models Outperform Large Language Modelscited by 15
  3. 2025Mano Technical Reportcited by 13
  4. 2025CompTrack: Information Bottleneck-Guided Low-Rank Dynamic Token Compression for Point Cloud Trackingcited by 9
  5. 2025SalQ-VLM: Fine-Grained Saliency-Guided Quantization for Vision-Language Modelscited by 8
  6. 2026VisNec: Measuring and Leveraging Visual Necessity for Multimodal Instruction Tuningcited by 8
  7. 2026RoboRouter: Training-Free Policy Routing for Robotic Manipulationcited by 6
  8. 2026BATQuant: Outlier-resilient MXFP4 Quantization via Learnable Block-wise Optimizationcited by 6
  9. 2025Towards Efficient Multi-LLM Inference: Characterization and Analysis of LLM Routing and Hierarchical Techniquescited by 5
  10. 2026MASQuant: Modality-Aware Smoothing Quantization for Multimodal Large Language Modelscited by 5
  11. 2025Beyond Frame-wise Tracking: A Trajectory-based Paradigm for Efficient Point Cloud Trackingcited by 4
  12. 2026VEQ: Modality-Adaptive Quantization for MoE Vision-Language Modelscited by 4
23 more
  1. 2026FreeAct: Freeing Activations for LLM Quantizationcited by 4
  2. 2025EffiVLM-BENCH: A Comprehensive Benchmark for Evaluating Training-Free Acceleration in Large Vision-Language Modelscited by 3
  3. 2025Diffusion-Based Image Editing: An Unforeseen Adversary to Robust Invisible Watermarkscited by 3
  4. 2025Revisiting Multimodal KV Cache Compression: A Frequency-Domain-Guided Outlier-KV-Aware Approachcited by 3
  5. 2026TORQ: Two-Level Orthogonal Rotation for MXFP4 Quantizationcited by 3
  6. 2025FQ-PETR: Fully Quantized Position Embedding Transformation for Multi-View 3D Object Detectioncited by 2
  7. 2026Rethinking Practical and Efficient Quantization Calibration for Vision-Language Modelscited by 2
  8. 2026Vanishing Watermarks: Diffusion-Based Image Editing Undermines Robust Invisible Watermarkingcited by 2
  9. 2026BlackMirror: Black-Box Backdoor Detection for Text-to-Image Models via Instruction-Response Deviationcited by 1
  10. 2026Editing on the Generative Manifold: A Theoretical and Empirical Study of General Diffusion-Based Image Editing Trade-offscited by 1
  11. 2026QAPruner: Quantization-Aware Vision Token Pruning for Multimodal Large Language Modelscited by 1
  12. 2026MGVQ: Synergizing Multi-dimensional Sensitivity-Aware and Gradient-Hessian Fusion for Vector Quantizationcited by 1
  13. 2026QVGGT: Post-Training Quantized Visual Geometry Grounded Transformercited by 1
  14. 2026BEFORE THE FLIP: Measuring Hidden Score Shifts In Quantized Vision Language Models Before The Answer Changes for Visual Question Answeringcited by 1
  15. 2025FQ-PETR: Fully Quantized Position Embedding Transformation for Multi-View 3D Object Detectioncited by 0
  16. 2025Beyond Real Weights: Hypercomplex Representations for Stable Quantizationcited by 0
  17. 2026When Denoising Becomes Unsigning: Theoretical and Empirical Analysis of Watermark Fragility Under Diffusion-Based Image Editingcited by 0
  18. 2026Editing Away the Evidence: Diffusion-Based Image Manipulation and the Failure Modes of Robust Watermarkingcited by 0
  19. 2026MorphoQuant: Modality-Aware Quantization for Omni-modal Large Language Modelscited by 0
  20. 2026On the Controllability-Fidelity Frontier in Diffusion Editingcited by 0
  21. 2026Activation Outliers Matter: Robust Recovery for Quantized Multimodal LLMscited by 0
  22. 2026MiX: Micro-Inverted-Scaling for End-to-End Low-Bit Vision-Language Model Accelerationcited by 0
  23. 2026RGSQ: Riemannian Geometry-Sensitive Quantization for Large Vision-Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Fast-Slow Efficient Training for Multimodal Large Language Models via Visual Token Pruning
  2. 2024Accelerating Multimodal Large Language Models via Dynamic Visual-Token Exit and the Empirical Findings
  3. 2024The Revolution of Multimodal Large Language Models: A Survey
  4. 2024Boosting Multimodal Large Language Models with Visual Tokens Withdrawal for Rapid Inference
  5. 2024ST3: Accelerating Multimodal Large Language Model by Spatial-Temporal Visual Token Trimming
  6. 2025Unleashing the Intrinsic Visual Representation Capability of Multimodal Large Language Models
  7. 2025SAISA: Towards Multimodal Large Language Models with Both Training and Inference Efficiency
  8. 2025LFTR: Learning-Free Token Reduction for Multimodal Large Language Models