mlsanityData to 7 Oct 2026

AirCache: Activating Inter-modal Relevancy KV Cache Compression for Efficient Large Vision-Language Model Inference

arXiv 2503.23956 ↗First posted 31 Mar 2025cs.CV cs.AIcited by 4

On the map: LLM reasoning → Multimodal reasoning

also in 2 more
Abstract (arXiv)

Recent advancements in Large Visual Language Models (LVLMs) have gained significant attention due to their remarkable reasoning capabilities and proficiency in generalization. However, processing a large number of visual tokens and generating long-context outputs impose substantial computational overhead, leading to excessive demands for key-value (KV) cache. To address this critical bottleneck, we propose AirCache, a novel KV cache compression method aimed at accelerating LVLMs inference. This work systematically investigates the correlations between visual and textual tokens within the attention mechanisms of LVLMs. Our empirical analysis reveals considerable redundancy in cached visual tokens, wherein strategically eliminating these tokens preserves model performance while significantly accelerating context generation. Inspired by these findings, we introduce an elite observation window for assessing the importance of visual components in the KV cache, focusing on stable inter-modal relevancy modeling with enhanced multi-perspective consistency. Additionally, we develop an adaptive layer-wise budget allocation strategy that capitalizes on the strength and skewness of token importance distribution, showcasing superior efficiency compared to uniform allocation. Comprehensive evaluations across multiple LVLMs and benchmarks demonstrate that our method achieves comparable performance to the full cache while retaining only 10% of visual KV cache, thereby reducing decoding latency by 29% to 66% across various batch size and prompt length of inputs. Notably, as cache retention rates decrease, our method exhibits increasing performance advantages over existing approaches.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 20 of its 42 references that are in the map. First posted on arXiv 31 Mar 2025.

Builds on

20 of 42 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024PrefixKV: Adaptive Prefix KV Cache is What Vision Instruction-Following Models Need for Efficient Generation4 Dec
  2. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning19 Mar

Other references · 18

  1. 2024Qwen2.5 Technical Reportcited by 4,713
  2. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  3. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  4. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  5. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 848
  6. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 559
  7. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 550
  8. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 420
  9. 2024MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understandingcited by 180
  10. 2024TokenPacker: Efficient Visual Projector for Multimodal LLMcited by 170
  11. 2024Beyond Text-Visual Attention: Exploiting Visual Cues for Effective Token Pruning in VLMscited by 153
  12. 2023MobileVLM : A Fast, Strong and Open Vision Language Assistant for Mobile Devicescited by 120
6 more
  1. 2024Dynamic Memory Compression: Retrofitting LLMs for Accelerated Inferencecited by 115
  2. 2024Matryoshka Query Transformer for Large Vision-Language Modelscited by 49
  3. 2024ZipVL: Efficient Large Vision-Language Models with Dynamic Token Sparsificationcited by 43
  4. 2023Compressed Context Memory For Online Language Model Interactioncited by 42
  5. 2024VL-Cache: Sparsity and Modality-Aware KV Cache Compression for Vision-Language Model Inference Accelerationcited by 41
  6. 2024KVSharer: Efficient Inference via Layer-Wise Dissimilar KV Cache Sharingcited by 38

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Why and When Visual Token Pruning Fails? A Study on Relevant Visual Information Shift in MLLMs Decodingcited by 6
  2. 2026KVCapsule: Efficient Sequential KV Cache Compression for Vision-Language Models with Asymmetric Redundancycited by 0
  3. 2026CIVIC: End-to-End Sequence Compactness for Efficient Vision-Language Modelscited by 0
  4. 2026Visual Memory Attacks Can Persist Through The KV Cachecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026VisCache: Visual KV Cache Pruning for Efficient Vision Large Language Model Inference
  2. 2026HybridKV: Hybrid KV Cache Compression for Efficient Multimodal Large Language Model Inference
  3. 2025Mixing Importance with Diversity: Joint Optimization for KV Cache Compression in Large Vision-Language Models
  4. 2026Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospects
  5. 2026TGV-KV: Text-Grounded KV Eviction for Vision-Language Models
  6. 2026Towards Efficient Large Vision-Language Models: A Comprehensive Survey on Inference Strategies
  7. 2026Make Your LVLM KV Cache More Lightweight
  8. 2026Reducing Peak Memory Usage for Modern Multimodal Large Language Model Pipelines