mlsanityData to 14 Oct 2026

MEDA: Dynamic KV Cache Allocation for Efficient Multimodal Long-Context Inference

arXiv 2502.17599 ↗First posted 24 Feb 2025cs.CLcited by 31

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

also in 1 more
Abstract (arXiv)

Long-context Multimodal Large Language Models (MLLMs) that incorporate long text-image and text-video modalities, demand substantial resources as their multimodal Key-Value (KV) caches grow with increasing input lengths, challenging inference efficiency. Existing methods for KV cache compression, in both text-only and multimodal LLMs, have neglected attention density variations across layers, thus often adopting uniform or progressive reduction strategies for layer-wise cache allocation. In this work, we propose MEDA, a dynamic layer-wise KV cache allocation method for efficient multimodal long-context inference. As its core, MEDA utilizes cross-modal attention entropy to determine the KV cache size at each MLLMs layer. Given the dynamically allocated KV cache size at each layer, MEDA also employs a KV pair selection scheme to identify which KV pairs to select and a KV pair merging strategy that merges the selected and non-selected ones to preserve information from the entire context. MEDA achieves up to 72% KV cache memory reduction and 2.82 times faster decoding speed, while maintaining or enhancing performance on various multimodal tasks in long-context settings, including multi-images and long-video scenarios. Our code is released at https://github.com/AIoT-MLSys-Lab/MEDA.

Key facts. Cited by 31 papers in the map (data through 14 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 35 of its 61 references that are in the map. First posted on arXiv 24 Feb 2025.

Builds on

35 of 61 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024LOOK-M: Look-Once Optimization in KV Cache for Efficient Multimodal Long-Context Inference26 Jun

Other references · 34

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  3. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  4. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  5. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 880
  6. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 832
  7. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 637
  8. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 583
  9. 2024Long Context Transfer from Language to Visioncited by 515
  10. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 432
  11. 2024LongVILA: Scaling Long-Context Visual Language Models for Long Videoscited by 309
  12. 2024SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compressioncited by 243
22 more
  1. 2024Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understandingcited by 213
  2. 2024World Model on Million-Length Video And Language With Blockwise RingAttentioncited by 198
  3. 2023Stabilizing Transformer Training by Preventing Attention Entropy Collapsecited by 161
  4. 2024GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLMcited by 158
  5. 2024PyramidInfer: Pyramid KV Cache Compression for High-throughput LLM Inferencecited by 147
  6. 2024Tarsier: Recipes for Training and Evaluating Large Video Description Modelscited by 141
  7. 2024Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabulariescited by 118
  8. 2024LongLLaVA: Scaling Multi-modal LLMs to 1000 Images Efficiently via a Hybrid Architecturecited by 102
  9. 2024LLaMA-Berry: Pairwise Optimization for O1-like Olympiad-Level Mathematical Reasoningcited by 91
  10. 2024MileBench: Benchmarking MLLMs in Long Contextcited by 88
  11. 2024MADTP: Multimodal Alignment-Guided Dynamic Token Pruning for Accelerating Vision-Language Transformercited by 54
  12. 2024Autoregressive Models in Vision: A Surveycited by 39
  13. 2025Recent Advances in Large Langauge Model Benchmarks against Data Contamination: From Static to Dynamic Evaluationcited by 38
  14. 2025Efficient Diffusion Models: A Surveycited by 35
  15. 2024On the Efficacy of Eviction Policy for Key-Value Constrained Generative Language Model Inferencecited by 32
  16. 2024MEIT: Multimodal Electrocardiogram Instruction Tuning on Large Language Models for Report Generationcited by 29
  17. 2024NeuroClips: Towards High-fidelity and Smooth fMRI-to-Video Reconstructioncited by 26
  18. 2024WorldQA: Multimodal World Knowledge in Videos through Long-Chain Reasoningcited by 16
  19. 2024UNComp: Can Matrix Entropy Uncover Sparsity? -- A Compressor Design from an Uncertainty-Aware Perspectivecited by 15
  20. 2021Alignment Attention by Matching Key and Query Distributionscited by 14
  21. 2024Evolver: Chain-of-Evolution Prompting to Boost Large Multimodal Models for Hateful Meme Detectioncited by 9
  22. 2024Famba-V: Fast Vision Mamba with Cross-Layer Token Fusioncited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 11; 2026: 20.

31 papers.

Sort:

By area:

  1. 2025SVD-LLM V2: Optimizing Singular Value Truncation for Large Language Model Compressioncited by 67
  2. 2025SRPO: Enhancing Multimodal LLM Reasoning via Reflection-Aware Reinforcement Learningcited by 57
  3. 2025Look-Back: Implicit Visual Re-focusing in MLLM Reasoningcited by 55
  4. 2025Shifting AI Efficiency From Model-Centric to Data-Centric Compressioncited by 29
  5. 2025Head-Aware KV Cache Compression for Efficient Visual Autoregressive Modelingcited by 15
  6. 2025GUI-KV: Efficient GUI Agents via KV Cache with Spatio-Temporal Awarenesscited by 11
  7. 2026Make Each Token Count: Towards Improving Long-Context Performance with KV Cache Evictioncited by 11
  8. 2026Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospectscited by 9
  9. 2025LightVLM: Acceleraing Large Multimodal Models with Pyramid Token Merging and KV Cache Compressioncited by 7
  10. 2026Event-VStream: Event-Driven Real-Time Understanding for Long Video Streamscited by 6
  11. 2025Enhancing Test-Time Scaling of Large Language Models with Hierarchical Retrieval-Augmented MCTScited by 5
  12. 2025Revisiting Multimodal KV Cache Compression: A Frequency-Domain-Guided Outlier-KV-Aware Approachcited by 3
19 more
  1. 2026HybridKV: Hybrid KV Cache Compression for Efficient Multimodal Large Language Model Inferencecited by 3
  2. 2026Prefill-Time Intervention for Mitigating Hallucination in Large Vision-Language Modelscited by 3
  3. 2026RetentiveKV: State-Space Memory for Uncertainty-Aware Multimodal KV Cache Evictioncited by 3
  4. 2026PRISM: Pareto-Efficient Retrieval over Intent-Aware Structured Memory for Long-Horizon Agentscited by 3
  5. 2025Adaptive and Resource-efficient Agentic AI Systems for Mobile and Embedded Devices: A Surveycited by 2
  6. 2026ST-Lite: Training-Free KV Cache Compression with Spatio-Trajectory Guidance for Long-Horizon GUI Agentscited by 2
  7. 2026ERA: Entropy-Guided Visual Token Pruning with Rectified Attention for Efficient MLLMscited by 2
  8. 2025Dynamic Token Reweighting for Robust Vision-Language Modelscited by 1
  9. 2026SpecEyes: Accelerating Agentic Multimodal LLMs via Speculative Perception and Planningcited by 1
  10. 2026Linear Scaling Video VLMs for Long Video Understandingcited by 1
  11. 2026STaR-KV: Spatio-Temporal Adaptive Re-weighting for KV Cache Compression in GUI Vision-Language Modelscited by 1
  12. 2026HACK++: Towards More Effective Head-Aware Key-Value Compression for Efficient Visual Autoregressive Modelingcited by 1
  13. 2026Reducing Peak Memory Usage for Modern Multimodal Large Language Model Pipelinescited by 0
  14. 2026Learning to See What You Need: Gaze Attention for Multimodal Large Language Modelscited by 0
  15. 2026VisCache: Visual KV Cache Pruning for Efficient Vision Large Language Model Inferencecited by 0
  16. 2026CONDUIT: A Unified Residual-Stream Restoration Framework for KV Cache Reuse in Vision-Language Modelscited by 0
  17. 2026ECOKV: Geometry-Aware KV Cache Eviction via Complementary Diversity Metricscited by 0
  18. 2026Why Is Video Still So Expensive? A Survey of Inference-Efficiency Mechanisms in Video and Audiovisual LLMscited by 0
  19. 2026OmniTide: Co-Designing Algorithms and Systems for Efficient On-Device Omni-LLM Streamingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025KVCompose: Efficient Structured KV Cache Compression with Composite Tokens
  2. 2026Last But Not Least: Boundary Attention CalibratiON for Multimodal KV Cache Compression
  3. 2025MadaKV: Adaptive Modality-Perception KV Cache Eviction for Efficient Multimodal Long-Context Inference
  4. 2026Hierarchical Adaptive Eviction for KV Cache Management in Multimodal Language Models
  5. 2025BaKlaVa -- Budgeted Allocation of KV cache for Long-context Inference
  6. 2025LLMs Know What to Drop: Self-Attention Guided KV Cache Eviction for Efficient Long-Context Inference
  7. 2025RocketKV: Accelerating Long-Context LLM Inference via Two-Stage KV Cache Compression
  8. 2026MM-ShiftKV: Decode-Aware Prefill-Stage KV Selection for Multimodal Large Language Models