mlsanityData to 14 Oct 2026

SAB-LVLM: Significance-Aware Binarization for Large Vision-Language Models

arXiv 2607.01876 ↗First posted 2 Jul 2026cs.CV cs.AIcited by 1

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

Abstract (arXiv)

Large Vision-Language Models (LVLMs) have achieved remarkable progress in multimodal understanding, yet their enormous parameter scale and cross-modal computation incur substantial memory and latency overhead, severely limiting real-world deployment on resource-constrained devices. Binarization offers an attractive solution by drastically reducing storage and computational costs. However, existing binarization methods neglect the varying importance of weights across different layers and modalities. This causes parameters irrelevant to downstream tasks to be unnecessarily retained, whereas modality-critical weights may not be adequately optimized, resulting in significant performance degradation. To address these challenges, we develop a novel underlineSignificance-underlineAware underlineBinarization for underlineLarge underlineVision-underlineLanguage underlineModels (SAB-LVLM). Specifically, after constructing Hessian matrices for textual and visual inputs, we propose a spatial significance map to distinguish full-precision weights activated under a single modality from those activated across modalities. We then devise a modality-guided integration strategy to obtain the significance-aware binarization map, which measures weight significance across layers and modalities. Subsequently, this binarization map is incorporated into the binarization objective as an error reweighting term, and binarization fitting is performed through an alternating significance-weighted update scheme. Extensive experiments illustrate the superiority of our SAB-LVLM over existing binary PTQ methods under an approximately 1-bit compression constraint. Our code is accessible at https://github.com/LyuQi127/SABLVLM.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 45 of its 53 references that are in the map. First posted on arXiv 2 Jul 2026.

Builds on

45 of 53 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024MBQ: Modality-Balanced Quantization for Large Vision-Language Models27 Dec

Other references · 44

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  6. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  7. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  8. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  9. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  10. 2019Towards VQA Models That Can Readcited by 1,718
  11. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  12. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
32 more
  1. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  2. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  3. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  4. 2016Modeling Context in Referring Expressionscited by 1,063
  5. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
  6. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 800
  7. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 611
  8. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 526
  9. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 479
  10. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 402
  11. 2016Visual Dialogcited by 377
  12. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 367
  13. 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 353
  14. 2023LLM-QAT: Data-Free Quantization Aware Training for Large Language Modelscited by 324
  15. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 294
  16. 2023QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Modelscited by 155
  17. 2024BiLLM: Pushing the Limit of Post-Training Quantization for LLMscited by 144
  18. 2023RPTQ: Reorder-based Post-training Quantization for Large Language Modelscited by 116
  19. 2023OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Modelscited by 114
  20. 2023PB-LLM: Partially Binarized Large Language Modelscited by 83
  21. 2023Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scalingcited by 61
  22. 2024Compressing Large Language Models using Low Rank and Low Precision Decompositioncited by 59
  23. 2024STBLLM: Breaking the 1-Bit Barrier with Structured Binary LLMscited by 31
  24. 2024ARB-LLM: Alternating Refined Binarizations for Large Language Modelscited by 31
  25. 2024DDK: Distilling Domain Knowledge for Efficient Large Language Modelscited by 26
  26. 2024Quantization without Tearscited by 25
  27. 2024DISP-LLM: Dimension-Independent Structural Pruning for Large Language Modelscited by 24
  28. 2024ReALFRED: An Embodied Instruction Following Benchmark in Photo-Realistic Environmentscited by 16
  29. 2024SKIM: Any-bit Quantization Pushing The Limits of Post-Training Quantizationcited by 6
  30. 2026SliderQuant: Accurate Post-Training Quantization for LLMscited by 5
  31. 2025HBLLM: Wavelet-Enhanced High-Fidelity 1-Bit Quantization for LLMscited by 4
  32. 2026Lifelong Embodied Navigation Learningcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026LEEVLA: Seeing What Matters in Latent Environment Evolution for Vision-Language-Actioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Towards Efficient Large Vision-Language Models: A Comprehensive Survey on Inference Strategies
  2. 2025LaVi: Efficient Large Vision-Language Models via Internal Feature Modulation
  3. 2026Fine-Grained Post-Training Quantization for Large Vision Language Models with Quantization-Aware Integrated Gradients
  4. 2024Q-VLM: Post-training Quantization for Large Vision-Language Models
  5. 2026A Strong Baseline for Evaluating Vision Encoders in Multimodal Large Language Models
  6. 2025Benchmarking Large Vision-Language Models on Fine-Grained Image Tasks: A Comprehensive Evaluation
  7. 2026Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospects
  8. 2024Dynamic-LLaVA: Efficient Multimodal Large Language Models via Dynamic Vision-language Context Sparsification