mlsanityData to 14 Oct 2026

MPIC: Position-Independent Multimodal Context Caching System for Efficient MLLM Serving

arXiv 2502.01960 ↗First posted 4 Feb 2025cs.LGcited by 8

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

also in 1 more
Abstract (arXiv)

The context caching technique is employed to accelerate the Multimodal Large Language Model (MLLM) inference by prevailing serving platforms currently. However, this approach merely reuses the Key-Value (KV) cache of the initial sequence of prompt, resulting in full KV cache recomputation even if the prefix differs slightly. This becomes particularly inefficient in the context of interleaved text and images, as well as multimodal retrieval-augmented generation. This paper proposes position-independent caching as a more effective approach for multimodal information management. We have designed and implemented a caching system, named MPIC, to address both system-level and algorithm-level challenges. MPIC stores the KV cache on local disks when receiving multimodal data, and calculates and loads the KV cache in parallel during inference. To mitigate accuracy degradation, we have incorporated the integrated reuse and recompute mechanism within the system. The experimental results demonstrate that MPIC can achieve up to 54% reduction in response time and 2× improvement in throughput compared to existing context caching systems, while maintaining negligible or no accuracy loss.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 36 of its 43 references that are in the map. First posted on arXiv 4 Feb 2025.

Builds on

36 of 43 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024A-VL: Adaptive Attention for Large Vision-Language Models23 Sep
  2. 2024An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models11 Mar

Other references · 34

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  4. 2020Longformer: The Long-Document Transformercited by 3,413
  5. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  6. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  7. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  8. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  9. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
  10. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  11. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 832
  12. 2023CogAgent: A Visual Language Model for GUI Agentscited by 704
22 more
  1. 2023In-Context Retrieval-Augmented Language Modelscited by 702
  2. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 617
  3. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 558
  4. 2022Efficiently Scaling Transformer Inferencecited by 539
  5. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 501
  6. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 451
  7. 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 338
  8. 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 268
  9. 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusioncited by 223
  10. 2023CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Servingcited by 203
  11. 2020Generation-Augmented Retrieval for Open-domain Question Answeringcited by 186
  12. 2023UniIR: Training and Benchmarking Universal Multimodal Information Retrieverscited by 166
  13. 2022A Survey on Retrieval-Augmented Text Generationcited by 158
  14. 2023S-LoRA: Serving Thousands of Concurrent LoRA Adapterscited by 152
  15. 2024Inference without Interference: Disaggregate LLM Inference for Mixed Downstream Workloadscited by 121
  16. 2024RAGCache: Efficient Knowledge Caching for Retrieval-Augmented Generationcited by 98
  17. 2024MMDU: A Multi-Turn Multi-Image Dialog Understanding Benchmark and Instruction-Tuning Dataset for LVLMscited by 90
  18. 2024MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Poolcited by 68
  19. 2023Stateful Large Language Model Serving with Pensievecited by 61
  20. 2024EPIC: Efficient Position-Independent Caching for Serving Large Language Modelscited by 47
  21. 2023Sparkles: Unlocking Chats Across Multiple Images for Multimodal Instruction-Following Modelscited by 28
  22. 2024CaraServe: CPU-Assisted and Rank-Aware LoRA Serving for Generative LLM Inferencecited by 22

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 6.

8 papers.

  1. 2025PrefillOnly: An Inference Engine for Prefill-only Workloads in Large Language Model Applicationscited by 21
  2. 2025VLCache: Computing 2% Vision Tokens and Reusing 98% for Vision-Language Inferencecited by 7
  3. 2026You Need an Encoder for Native Position-Independent Cachingcited by 6
  4. 2026Models Take Notes at Prefill: KV Cache Can Be Editable and Composablecited by 6
  5. 2026Functional Cache Grafting: Robust and Rapid Code-Policy Synthesis for Embodied Agentscited by 0
  6. 2026QV-PIC: Query-Aware Visual Position-Independent Caching for Efficient RAG Servingcited by 0
  7. 2026CONDUIT: A Unified Residual-Stream Restoration Framework for KV Cache Reuse in Vision-Language Modelscited by 0
  8. 2026ATTUNER: Recomputation-Free KV Cache Reuse via Query-Side Adaptationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024LOOK-M: Look-Once Optimization in KV Cache for Efficient Multimodal Long-Context Inference
  2. 2026Reducing Peak Memory Usage for Modern Multimodal Large Language Model Pipelines
  3. 2025MEDA: Dynamic KV Cache Allocation for Efficient Multimodal Long-Context Inference
  4. 2024Inf-MLLM: Efficient Streaming Inference of Multimodal Large Language Models on a Single GPU
  5. 2024A Survey on Large Language Model Acceleration based on KV Cache Management
  6. 2026Hierarchical Adaptive Eviction for KV Cache Management in Multimodal Language Models
  7. 2026HybridKV: Hybrid KV Cache Compression for Efficient Multimodal Large Language Model Inference
  8. 2026MM-ShiftKV: Decode-Aware Prefill-Stage KV Selection for Multimodal Large Language Models