mlsanityData to 14 Oct 2026

MadaKV: Adaptive Modality-Perception KV Cache Eviction for Efficient Multimodal Long-Context Inference

arXiv 2506.15724 ↗First posted 6 Jun 2025cs.LG cs.AI cs.CLcited by 12

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

also in 1 more
Abstract (arXiv)

This paper introduces MadaKV, a modality-adaptive key-value (KV) cache eviction strategy designed to enhance the efficiency of multimodal large language models (MLLMs) in long-context inference. In multimodal scenarios, attention heads exhibit varying preferences for different modalities, resulting in significant disparities in modality importance across attention heads. Traditional KV cache eviction methods, which are tailored for unimodal settings, fail to capture modality-specific information, thereby yielding suboptimal performance. MadaKV addresses these challenges through two key components: modality preference adaptation and hierarchical compression compensation. By dynamically sensing modality information within attention heads and adaptively retaining critical tokens, MadaKV achieves substantial reductions in KV cache memory footprint and model inference decoding latency (1.3 to 1.5 times improvement) while maintaining high accuracy across various multimodal long-context tasks. Extensive experiments on representative MLLMs and the MileBench benchmark demonstrate the effectiveness of MadaKV compared to existing KV cache eviction methods.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 33 of its 50 references that are in the map. First posted on arXiv 6 Jun 2025.

Builds on

33 of 50 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024LOOK-M: Look-Once Optimization in KV Cache for Efficient Multimodal Long-Context Inference26 Jun

Other references · 32

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  5. 2020Longformer: The Long-Document Transformercited by 3,413
  6. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  7. 2020Rethinking Attention with Performerscited by 1,601
  8. 2019Generating Long Sequences with Sparse Transformerscited by 1,546
  9. 2020Linformer: Self-Attention with Linear Complexitycited by 1,474
  10. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  11. 2016Training Deep Nets with Sublinear Memory Costcited by 848
  12. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 783
20 more
  1. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 591
  2. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 575
  3. 2023Model Tells You What to Discard: Adaptive KV Cache Compression for LLMscited by 474
  4. 2023BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editingcited by 442
  5. 2023Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Timecited by 439
  6. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 432
  7. 2021ZeRO-Offload: Democratizing Billion-Scale Model Trainingcited by 341
  8. 2024STAR: A Benchmark for Situated Reasoning in Real-World Videoscited by 213
  9. 2020MovieNet: A Holistic Dataset for Movie Understandingcited by 206
  10. 2024World Model on Million-Length Video And Language With Blockwise RingAttentioncited by 198
  11. 2023Focused Transformer: Contrastive Training for Context Scalingcited by 133
  12. 2018Learning to Describe Differences Between Pairs of Similar Imagescited by 114
  13. 2024No Token Left Behind: Reliable KV Cache Compression via Importance-Aware Mixed Precision Quantizationcited by 101
  14. 2024Keep the Cost Down: A Review on Methods to Optimize LLM' s KV-Cache Consumptioncited by 97
  15. 2024MileBench: Benchmarking MLLMs in Long Contextcited by 88
  16. 2019Explicit Sparse Transformer: Concentrated Attention Through Explicit Selectioncited by 62
  17. 2024LoRC: Low-Rank Compression for LLMs KV Cache with a Progressive Compression Strategycited by 51
  18. 2019Expressing Visual Relationships via Languagecited by 49
  19. 2025Disentangled Knowledge Tracing for Alleviating Cognitive Biascited by 12
  20. 2025Cuff-KT: Tackling Learners' Real-time Learning Pattern Adjustment via Tuning-Free Knowledge State Guided Model Updatingcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 11.

12 papers.

  1. 2026Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospectscited by 9
  2. 2026Not Just What's There: Enabling CLIP to Comprehend Negated Visual Descriptions Without Fine-tuningcited by 5
  3. 2026HybridKV: Hybrid KV Cache Compression for Efficient Multimodal Large Language Model Inferencecited by 3
  4. 2026Sub-Token Routing for KV Cache Compressioncited by 3
  5. 2026RetentiveKV: State-Space Memory for Uncertainty-Aware Multimodal KV Cache Evictioncited by 3
  6. 2025Collaborative Representation Learning for Alignment of Tactile, Language, and Vision Modalitiescited by 2
  7. 2026Multi-turn Conversational AI from Text to Multimodal Interaction: Data, Models, Evaluation, and Open Challengescited by 1
  8. 2026FairFS: Addressing Deep Feature Selection Biases for Recommender Systemcited by 0
  9. 2026KVCapsule: Efficient Sequential KV Cache Compression for Vision-Language Models with Asymmetric Redundancycited by 0
  10. 2026Beyond Independent Optimization: Compression, MoE Routing, and Quantization Interactions in Multimodal Edge Intelligencecited by 0
  11. 2026VisCache: Visual KV Cache Pruning for Efficient Vision Large Language Model Inferencecited by 0
  12. 2026ECOKV: Geometry-Aware KV Cache Eviction via Complementary Diversity Metricscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025MEDA: Dynamic KV Cache Allocation for Efficient Multimodal Long-Context Inference
  2. 2025AhaKV: Adaptive Holistic Attention-Driven KV Cache Eviction for Efficient Inference of Large Language Models
  3. 2026Hierarchical Adaptive Eviction for KV Cache Management in Multimodal Language Models
  4. 2026Reducing Peak Memory Usage for Modern Multimodal Large Language Model Pipelines
  5. 2024VL-Cache: Sparsity and Modality-Aware KV Cache Compression for Vision-Language Model Inference Acceleration
  6. 2025FlowMM: Cross-Modal Information Flow Guided KV Cache Merging for Efficient Multimodal Context Inference
  7. 2025AirCache: Activating Inter-modal Relevancy KV Cache Compression for Efficient Large Vision-Language Model Inference
  8. 2026TGV-KV: Text-Grounded KV Eviction for Vision-Language Models