mlsanityData to 14 Oct 2026

InfiniPot-V: Memory-Constrained KV Cache Compression for Streaming Video Understanding

arXiv 2506.15745 ↗First posted 18 Jun 2025eess.IV cs.LGcited by 58

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

Modern multimodal large language models (MLLMs) can reason over hour-long video, yet their key-value (KV) cache grows linearly with time-quickly exceeding the fixed memory of phones, AR glasses, and edge robots. Prior compression schemes either assume the whole video and user query are available offline or must first build the full cache, so memory still scales with stream length. InfiniPot-V is the first training-free, query-agnostic framework that enforces a hard, length-independent memory cap for streaming video understanding. During video encoding it monitors the cache and, once a user-set threshold is reached, runs a lightweight compression pass that (i) removes temporally redundant tokens via Temporal-axis Redundancy (TaR) metric and (ii) keeps semantically significant tokens via Value-Norm (VaN) ranking. Across four open-source MLLMs and four long-video and streaming-video benchmarks, InfiniPot-V cuts peak GPU memory by up to 94%, sustains real-time generation, and matches or surpasses full-cache accuracy-even in multi-turn dialogues. By dissolving the KV cache bottleneck without retraining or query knowledge, InfiniPot-V closes the gap for on-device streaming video assistants.

Key facts. Cited by 58 papers in the map (data through 14 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 33 of its 47 references that are in the map. First posted on arXiv 18 Jun 2025.

Builds on

33 of 47 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025KeyDiff: Key Similarity-Based KV Cache Eviction for Long-Context LLM Inference in Resource-Constrained Environments21 Apr
  2. 2024Not All Heads Matter: A Head-Level KV Cache Compression Method with Integrated Retrieval and Reasoning25 Oct
  3. 2024Locret: Enhancing Eviction in Long-Context LLM Inference with Trained Retaining Heads on Consumer-Grade Devices2 Oct
  4. 2024ThinK: Thinner Key Cache by Query-Driven Pruning30 Jul
  5. 2024RazorAttention: Efficient KV Cache Compression Through Retrieval Heads22 Jul
  6. 2024A Simple and Effective L2 Norm-Based Strategy for KV Cache Compression17 Jun
  7. 2024SnapKV: LLM Knows What You are Looking for Before Generation22 Apr
  8. 2023Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs3 Oct
  9. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models24 Jun

Other references · 24

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023Visual Instruction Tuningcited by 7,869
  3. 2024Qwen2.5 Technical Reportcited by 4,942
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  5. 2020Longformer: The Long-Document Transformercited by 3,413
  6. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  7. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  8. 2018SlowFast Networks for Video Recognitioncited by 1,867
  9. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  10. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  11. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
  12. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 665
12 more
  1. 2024An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Modelscited by 642
  2. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 601
  3. 2023Ring Attention with Blockwise Transformers for Near-Infinite Contextcited by 457
  4. 2024LongVILA: Scaling Long-Context Visual Language Models for Long Videoscited by 309
  5. 2020MovieNet: A Holistic Dataset for Movie Understandingcited by 206
  6. 2024StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understandingcited by 144
  7. 2024Towards Interpreting Visual Information Processing in Vision-Language Modelscited by 122
  8. 2025Streaming Video Question-Answering with In-context Video KV-Cache Retrievalcited by 117
  9. 2024Squeezed Attention: Accelerating Long Context Length LLM Inferencecited by 57
  10. 2025SlowFast-LLaVA-1.5: A Family of Token-Efficient Video Large Language Models for Long-Form Video Understandingcited by 30
  11. 2024InfiniPot: Infinite Context Processing on Memory-Constrained LLMscited by 27
  12. 2025Beyond RAG: Task-Aware KV Cache Compression for Comprehensive Knowledge Reasoningcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 51.

58 papers.

Sort:

By area:

  1. 2025StreamMem: Query-Agnostic KV Cache Memory for Streaming Video Understandingcited by 58
  2. 2025StreamingTOM: Streaming Token Compression for Efficient Video Understandingcited by 43
  3. 2026HERMES: KV Cache as Hierarchical Memory for Efficient Streaming Video Understandingcited by 41
  4. 2025StreamAgent: Towards Anticipatory Agents for Streaming Video Understandingcited by 33
  5. 2026Efficient Autoregressive Video Diffusion with Dummy Headcited by 20
  6. 2026StreamReady: Learning What to Answer and When in Long Streaming Videoscited by 19
  7. 2025Mixing Importance with Diversity: Joint Optimization for KV Cache Compression in Large Vision-Language Modelscited by 14
  8. 2026Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospectscited by 9
  9. 2026StreamMeCo: Long-Term Agent Memory Compression for Efficient Streaming Video Understandingcited by 9
  10. 2026Event-VStream: Event-Driven Real-Time Understanding for Long Video Streamscited by 6
  11. 2026MuKV: Multi-Grained KV Cache Compression for Long Streaming Video Question-Answeringcited by 6
  12. 2026OVO-S-Bench: A Hierarchical Benchmark for Streaming Spatial Intelligence in Multimodal LLMscited by 6
38 more
  1. 2025V-Rex: Real-Time Streaming Video LLM Acceleration via Dynamic KV Cache Retrievalcited by 5
  2. 2026Vista: Scene-Aware Optimization for Streaming Video Question Answering under Post-Hoc Queriescited by 5
  3. 2026CurveStream: Boosting Streaming Video Understanding in MLLMs via Curvature-Aware Hierarchical Visual Memory Managementcited by 5
  4. 2026Head Forcing: Long Autoregressive Video Generation via Head Heterogeneitycited by 5
  5. 2026EGOSTREAM: A Diagnostic Benchmark for Streaming Episodic Memory in Egocentric Visioncited by 5
  6. 2026StreamTTT: Reconciling Real-Time Perception and Long-Term Memory in Streaming VLMscited by 5
  7. 2025video-SALMONN S: Memory-Enhanced Streaming Audio-Visual LLMcited by 4
  8. 2026OmniMem: Perturbation-aware Memory Compression for Streaming Audio-Visual LLMscited by 4
  9. 2026Keep It in Mind: User Centric Continual Spatial Intelligence Reasoning in Egocentric Video Streamscited by 4
  10. 2026FOLIO: Focused Semantic Memory for Streaming Video Understandingcited by 4
  11. 2026TIR-Flow: Active Video Search and Reasoning with Frozen VLMscited by 3
  12. 2026EMFormer: Efficient Multi-Scale Transformer for Accumulative Context Weather Forecastingcited by 3
  13. 2026WeaveTime: Stream from Earlier Frames into Emergent Memory in VideoLLMscited by 3
  14. 2026FreshMem: Brain-Inspired Frequency-Space Hybrid Memory for Streaming Video Understandingcited by 2
  15. 2026Decouple and Cache: KV Cache Construction for Streaming Video Understandingcited by 2
  16. 2026ShallowStream: Index Shallow then Answer Deep for Streaming Video Understandingcited by 2
  17. 2025SFHand: Learning Embodied Manipulation by Streaming Egocentric 3D Hand Forecastingcited by 1
  18. 2026Going Down Memory Lane: Scaling Tokens for Video Stream Understanding with Dynamic KV-Cache Memorycited by 1
  19. 2026CoRDS: Coreset-based Representative and Diverse Selection for Streaming Video Understandingcited by 1
  20. 2026Linear Scaling Video VLMs for Long Video Understandingcited by 1
  21. 2026Last But Not Least: Boundary Attention CalibratiON for Multimodal KV Cache Compressioncited by 1
  22. 2026How Well Can Your Video Model Remember? Measuring Memory-Budget Trade-offs in Long Video Understandingcited by 1
  23. 2026ProtoKV: Streaming Video Understanding under Delayed Query with Summary-State Memorycited by 1
  24. 2026Light-Omni: Reflex over Reasoning in Agentic Video Understanding with Long-Term Memorycited by 1
  25. 2026StreamFlow: Dynamic Memory Flows for Streaming Video Understandingcited by 1
  26. 2026Beyond Retrieval: Progressive Latent Memory Evolution for Streaming Video Understandingcited by 1
  27. 2026Learning to See What You Need: Gaze Attention for Multimodal Large Language Modelscited by 0
  28. 2026Inverting the Streaming-Diffusion Bottleneck: Video-Rate MLLM-Conditioned Edit Diffusion on a Consumer GPUcited by 0
  29. 2026video-SALMONN-R3: Learning to ReWatch, ReAsk, and ReAnswer for Efficient Video Understandingcited by 0
  30. 2026QSVideo: Query-Conditioned Semantic Temporal Retrieval for Video Understandingcited by 0
  31. 2026Statistical Adversaries: Natural Backdoor-like Adversarial Features in Clean Vision Datasetscited by 0
  32. 2026SLVMBench: Skill Learning from Video Memorycited by 0
  33. 2026ECOKV: Geometry-Aware KV Cache Eviction via Complementary Diversity Metricscited by 0
  34. 2026Companion-style QA Assistance in Ego-Visioncited by 0
  35. 2026ReactVAU: A Slow-Fast Decoupled Framework for Streaming Video Anomaly Understandingcited by 0
  36. 2026Why Is Video Still So Expensive? A Survey of Inference-Efficiency Mechanisms in Video and Audiovisual LLMscited by 0
  37. 2026PREM: Prefix-Steered Recurrent Memory for Long-Video Understandingcited by 0
  38. 2026DeltaS: Reading the Gated Linear Attention State for KV Cache Eviction in Streaming Videocited by 0

50 most cited of 58 All 58 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Task-Aware KV Compression For Cost-Effective Long Video Understanding
  2. 2026MosaicKV: Serving Long-Context LLM with Dynamic Two-D KV Cache Compression
  3. 2025BaKlaVa -- Budgeted Allocation of KV cache for Long-context Inference
  4. 2025LagKV: Lag-Relative Information of the KV Cache Tells Which Tokens Are Important
  5. 2026VQKV: High-Fidelity and High-Ratio Cache Compression via Vector-Quantization
  6. 2024VL-Cache: Sparsity and Modality-Aware KV Cache Compression for Vision-Language Model Inference Acceleration
  7. 2026Make Your LVLM KV Cache More Lightweight
  8. 2026KV Cache Offloading for Context-Intensive Tasks