mlsanityData to 14 Oct 2026

QMoP: Query Guided Mixture-of-Projector for Efficient Visual Token Compression

arXiv 2603.21232 ↗First posted 22 Mar 2026cs.CV cs.AIcited by 3

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

also in 1 more
Abstract (arXiv)

Multimodal large language models suffer from severe computational and memory bottlenecks, as the number of visual tokens far exceeds that of textual tokens. While recent methods employ projector modules to align and compress visual tokens into text-aligned features, they typically depend on fixed heuristics that limit adaptability across diverse scenarios. In this paper, we first propose Query Guided Mixture-of-Projector (QMoP), a novel and flexible framework that adaptively compresses visual tokens via three collaborative branches: (1) a pooling-based branch for coarse-grained global semantics, (2) a resampler branch for extracting high-level semantic representations, and (3) a pruning-based branch for fine-grained token selection to preserve critical visual detail. To adaptively coordinate these branches, we introduce the Query Guided Router (QGR), which dynamically selects and weights the outputs from different branches based on both visual input and textual queries. A Mixture-of-Experts-style fusion mechanism is designed to aggregate the outputs, harnessing the strengths of each strategy while suppressing noise. To systematically evaluate the effects of Visual Token Compression, we also develop VTCBench, a dedicated benchmark for evaluating the information loss induced by visual token compression. Extensive experiments demonstrate that despite relying on fundamental compression modules, QMoP outperforms strong baselines and delivers significant savings in memory, computation, and inference time.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 38 of its 43 references that are in the map. First posted on arXiv 22 Mar 2026.

Builds on

38 of 43 references.

Closest ancestors · 15

Same direction, newest first

  1. 2025SCOPE: Saliency-Coverage Oriented Token Pruning for Efficient Multimodel LLMs28 Oct
  2. 2025Efficient Multi-modal Large Language Models via Progressive Consistency Distillation1 Oct
  3. 2025AutoPrune: Each Complexity Deserves a Pruning Policy28 Sep
  4. 2025Stop Looking for Important Tokens in Multimodal Language Models: Duplication Matters More17 Feb
  5. 2025LLaVA-Mini: Efficient Image and Video Large Multimodal Models with One Vision Token7 Jan
  6. 2024VisionZip: Longer is Better but Not Necessary in Vision Language Models5 Dec
  7. 2024Multi-Stage Vision Token Dropping: Towards Efficient Multimodal Large Language Model16 Nov
  8. 2024PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reduction22 Oct
  9. 2024SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference6 Oct
  10. 2024Fit and Prune: Fast and Training-free Visual Token Pruning for Multi-modal Large Language Models16 Sep
  11. 2024TokenPacker: Efficient Visual Projector for Multimodal LLM2 Jul
  12. 2024Matryoshka Query Transformer for Large Vision-Language Models29 May
3 more
  1. 2024Boosting Multimodal Large Language Models with Visual Tokens Withdrawal for Rapid Inference9 May
  2. 2024LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Models22 Mar
  3. 2024An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models11 Mar

Other references · 23

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2023Visual Instruction Tuningcited by 7,869
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  5. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  6. 2023Qwen Technical Reportcited by 3,344
  7. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  8. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  9. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  10. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  11. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  12. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
11 more
  1. 2019Towards VQA Models That Can Readcited by 1,718
  2. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  3. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
  4. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
  5. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 940
  6. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
  7. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 512
  8. 2023Honeybee: Locality-enhanced Projector for Multimodal LLMcited by 197
  9. 2024MobileVLM V2: Faster and Stronger Baseline for Vision Language Modelcited by 174
  10. 2023Fine-tuning Multimodal LLMs to Follow Zero-shot Demonstrative Instructionscited by 77
  11. 2024MME-Survey: A Comprehensive Survey on Evaluation of Multimodal LLMscited by 72

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Sub-Token Routing for KV Cache Compressioncited by 3
  2. 2026PARCEL: Pool-Anchored Resampling with Conditioned Elastic Queries for Efficient Vision-Language Understandingcited by 1
  3. 2026Beyond One-Size-Fits-All: Sample-Adaptive Strategy Routing for Vision Token Pruning in MLLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024DeCo: Decoupling Token Compression from Semantic Abstraction in Multimodal Large Language Models
  2. 2025Token Sequence Compression for Efficient Multimodal Computing
  3. 2025FlashVLM: Text-Guided Visual Token Selection for Large Multimodal Models
  4. 2026EvoComp: Learning Visual Token Compression for Multimodal Large Language Models via Semantic-Guided Evolutionary Labeling
  5. 2024FocusLLaVA: A Coarse-to-Fine Approach for Efficient and Effective Visual Token Compression
  6. 2026Balancing Saliency and Coverage: Semantic Prominence-Aware Budgeting for Visual Token Compression in VLMs
  7. 2025TokenCarve: Information-Preserving Visual Token Compression in Multimodal Large Language Models
  8. 2025PPE: Positional Preservation Embedding for Token Compression in Multimodal Large Language Models