mlsanityData to 14 Oct 2026

Streamline Without Sacrifice -- Squeeze out Computation Redundancy in LMM

arXiv 2505.15816 ↗First posted 21 May 2025cs.CVcited by 2

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

also in 1 more
Abstract (arXiv)

Large multimodal models excel in multimodal tasks but face significant computational challenges due to excessive computation on visual tokens. Unlike token reduction methods that focus on token-level redundancy, we identify and study the computation-level redundancy on vision tokens to ensure no information loss. Our key insight is that vision tokens from the pretrained vision encoder do not necessarily require all the heavy operations (e.g., self-attention, FFNs) in decoder-only LMMs and could be processed more lightly with proper designs. We designed a series of experiments to discover and progressively squeeze out the vision-related computation redundancy. Based on our findings, we propose ProxyV, a novel approach that utilizes proxy vision tokens to alleviate the computational burden on original vision tokens. ProxyV enhances efficiency without compromising performance and can even yield notable performance gains in scenarios with more moderate efficiency improvements. Furthermore, the flexibility of ProxyV is demonstrated through its combination with token reduction methods to boost efficiency further. The code will be made public at this https://github.com/penghao-wu/ProxyV URL.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 57 of its 64 references that are in the map. First posted on arXiv 21 May 2025.

Builds on

57 of 64 references.

Closest ancestors · 12

Same direction, newest first

  1. 2024Feather the Throttle: Revisiting Visual Token Pruning for Vision-Language Model Acceleration17 Dec
  2. 2024VisionZip: Longer is Better but Not Necessary in Vision Language Models5 Dec
  3. 2024AIM: Adaptive Inference of Multi-Modal LLMs via Token Merging and Pruning4 Dec
  4. 2024Accelerating Multimodal Large Language Models by Searching Optimal Vision Token Reduction30 Nov
  5. 2024FoPru: Focal Pruning for Efficient Large Vision-Language Models21 Nov
  6. 2024PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reduction22 Oct
  7. 2024ZipVL: Efficient Large Vision-Language Models with Dynamic Token Sparsification11 Oct
  8. 2024SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference6 Oct
  9. 2024Efficient Inference of Vision Instruction-Following Models with Elastic Cache25 Jul
  10. 2024LOOK-M: Look-Once Optimization in KV Cache for Efficient Multimodal Long-Context Inference26 Jun
  11. 2024LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Models22 Mar
  12. 2024An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models11 Mar

Other references · 45

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  6. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  7. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  8. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  9. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  10. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  11. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  12. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
33 more
  1. 2024Qwen2 Technical Reportcited by 2,120
  2. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  3. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  4. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  5. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
  6. 2019Towards VQA Models That Can Readcited by 1,718
  7. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  8. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  9. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
  10. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  11. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
  12. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,005
  13. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 940
  14. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
  15. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
  16. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 832
  17. 2016A Diagram Is Worth A Dozen Imagescited by 804
  18. 2023VILA: On Pre-training for Visual Language Modelscited by 692
  19. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 584
  20. 2024Long Context Transfer from Language to Visioncited by 515
  21. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 476
  22. 2023Model Tells You What to Discard: Adaptive KV Cache Compression for LLMscited by 474
  23. 2021InfographicVQAcited by 437
  24. 2024InternLM2 Technical Reportcited by 397
  25. 2024LongVILA: Scaling Long-Context Visual Language Models for Long Videoscited by 309
  26. 2024mPLUG-DocOwl 1.5: Unified Structure Learning for OCR-free Document Understandingcited by 220
  27. 2024NVLM: Open Frontier-Class Multimodal LLMscited by 125
  28. 2024Benchmarking and Improving Detail Image Captioncited by 81
  29. 2023CCpdf: Building a High Quality Corpus for Visually Rich Documents from Web Crawl Datacited by 21
  30. 2024Deciphering Cross-Modal Alignment in Large Vision-Language Models with Modality Integration Ratecited by 20
  31. 2024Cross-Self KV Cache Pruning for Efficient Vision-Language Inferencecited by 12
  32. 2024EE-MLLM: A Data-Efficient and Compute-Efficient Multimodal Large Language Modelcited by 10
  33. 2024Libra: Building Decoupled Vision System on Large Language Modelscited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodalitycited by 50
  2. 2026Efficient Agentic Reasoning Through Self-Regulated Simulative Planningcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024iLLaVA: An Image is Worth Fewer Than 1/3 Input Tokens in Large Multimodal Models
  2. 2025Variation-aware Vision Token Dropping for Faster Large Vision-Language Models
  3. 2025LLaVA-Mini: Efficient Image and Video Large Multimodal Models with One Vision Token
  4. 2025Token Sequence Compression for Efficient Multimodal Computing
  5. 2024VideoLLM-MoD: Efficient Video-Language Streaming with Mixture-of-Depths Vision Computation
  6. 2025Learning Compact Vision Tokens for Efficient Large Multimodal Models
  7. 2026Vision Token Reduction via Attention-Driven Self-Compression for Efficient Multimodal Large Language Models
  8. 2025Can Visual Input Be Compressed? A Visual Token Compression Benchmark for Large Multimodal Models