mlsanityData to 14 Oct 2026

Kamera: Unified Position-Invariant Multimodal KV Cache for Training-Free Reuse

arXiv 2606.23581 ↗First posted 22 Jun 2026cs.DC cs.AI cs.CVcited by 3

On the map: Efficient LLM inference and serving → Visual token pruning in multimodal models

Abstract (arXiv)

Multimodal agents repeatedly re-examine the same video frames, UI screenshots, and rendered artifacts as their context window slides and reasoning iterates, yet every look-back re-encodes from scratch, because prefix caches serve reuse only at a fixed leading position. We show this recompute is avoidable, and identify exactly what naive KV reuse loses: the cross-chunk conditioning a chunk absorbs from its neighbours. This loss is asymmetric. The direct readout of a cached chunk is recovered exactly and for free by the standard state-merge. What remains is a diffuse, low-rank residue concentrated in deep layers, invisible to single-hop retrieval but precisely what multi-hop reasoning binds on. Blind reuse therefore leaves single-hop recall intact while halving multi-hop accuracy; this is the failure mode prior position-independent caches, designed for single-context or single-image reuse, do not address. We repair it with a small, training-free low-rank conditioning patch stored alongside each position-free chunk. Reuse reduces to one operator across MLA, GQA, and MHA: exact RoPE re-rotation to any target position, plus the patch that restores cross-chunk binding. This makes three window operations cheap: reorder (one patch serves every ordering of a cached set), sliding-window survival (surviving chunks relocate via rotation only, zero re-encode), and recall (an evicted chunk is rehydrated by its patch, never re-encoded). A rank-m patch recovers full task accuracy on cross-chunk-binding benchmarks, MM-NIAH across two attention families and two-page doc-QA, at a fraction of the KV footprint, and reconstructs re-prefill KV to within bf16 rounding in a production SGLang kernel across six backbones. The conditioning signal is strongest in redundant vision and video streams, making our solution most impactful where multimodal agents spend their recompute budget.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Visual token pruning in multimodal models, one of 12 directions in Efficient LLM inference and serving. Builds on 45 of its 47 references that are in the map. First posted on arXiv 22 Jun 2026.

Builds on

45 of 47 references.

Closest ancestors · 2

Same direction, newest first

  1. 2026HERMES: KV Cache as Hierarchical Memory for Efficient Streaming Video Understanding21 Jan
  2. 2025StreamMem: Query-Agnostic KV Cache Memory for Streaming Video Understanding21 Aug

Other references · 43

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  2. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  4. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  5. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  6. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  7. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  8. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  9. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  10. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  11. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
  12. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
31 more
  1. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 832
  2. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
  3. 2024GPT-4V(ision) is a Generalist Web Agent, if Groundedcited by 563
  4. 2024FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisioncited by 519
  5. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 501
  6. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
  7. 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Modelscited by 438
  8. 2025SmolVLM: Redefining small and efficient multimodal modelscited by 237
  9. 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusioncited by 223
  10. 2024MuirBench: A Comprehensive Benchmark for Robust Multi-image Understandingcited by 188
  11. 2024GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLMcited by 158
  12. 2024ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inferencecited by 134
  13. 2024M3DocRAG: Multi-modal Retrieval is What You Need for Multi-page Multi-document Understandingcited by 111
  14. 2024ZoomEye: Enhancing Multimodal LLMs with Human-Like Zooming Capabilities through Tree-Based Image Explorationcited by 103
  15. 2025Deep Video Discovery: Agentic Search with Tool Use for Long-form Video Understandingcited by 98
  16. 2024LOOK-M: Look-Once Optimization in KV Cache for Efficient Multimodal Long-Context Inferencecited by 95
  17. 2025StreamingVLM: Real-Time Understanding for Infinite Video Streamscited by 95
  18. 2024MileBench: Benchmarking MLLMs in Long Contextcited by 88
  19. 2025VLA-Cache: Efficient Vision-Language-Action Manipulation via Adaptive Token Cachingcited by 86
  20. 2025KVLink: Accelerating Large Language Models via Efficient KV Cache Reusecited by 66
  21. 2024Needle In A Multimodal Haystackcited by 58
  22. 2024EPIC: Efficient Position-Independent Caching for Serving Large Language Modelscited by 47
  23. 2025LOVE-R1: Advancing Long Video Understanding with an Adaptive Zoom-in Mechanism via Multi-Step Reasoningcited by 27
  24. 2024Embodied VideoAgent: Persistent Memory from Egocentric Videos and Embodied Sensors Enables Dynamic Scene Understandingcited by 24
  25. 2025CacheClip: Accelerating RAG with Effective KV Cache Reusecited by 16
  26. 2025PixelCraft: A Multi-Agent System for High-Fidelity Visual Reasoning on Structured Imagescited by 12
  27. 2025PAL-UI: Planning with Active Look-back for Vision-Based GUI Agentscited by 11
  28. 2025VLCache: Computing 2% Vision Tokens and Reusing 98% for Vision-Language Inferencecited by 7
  29. 2026KEEP: A KV-Cache-Centric Memory Management System for Efficient Embodied Planningcited by 5
  30. 2026Semantic Cache Distillation: Efficient State Transfer via Reuse and Selective Patchingcited by 5
  31. 2026Doc-V*:Coarse-to-Fine Interactive Visual Reasoning for Multi-Page Document VQAcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Position Rebinding Cache Reuse: Replay-Free Visual Revisiting for Interleaved Multimodal Reasoning
  2. 2026Latent Visual Cache for Video Reasoning
  3. 2026UniCache: Task- and Type-Aware KV Cache Compression for Unified Multimodal Models
  4. 2025AirCache: Activating Inter-modal Relevancy KV Cache Compression for Efficient Large Vision-Language Model Inference
  5. 2026CONDUIT: A Unified Residual-Stream Restoration Framework for KV Cache Reuse in Vision-Language Models
  6. 2026RetentiveKV: State-Space Memory for Uncertainty-Aware Multimodal KV Cache Eviction
  7. 2026Neural Introspection Gating for Adaptive KV-Cache Reuse in Vision-Language-Action Models
  8. 2026ResAdapt: Adaptive Resolution for Efficient Multimodal Reasoning