mlsanityData to 14 Oct 2026

StreamMem: Query-Agnostic KV Cache Memory for Streaming Video Understanding

arXiv 2508.15717 ↗First posted 21 Aug 2025cs.CV cs.AIcited by 58

On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs

also in 1 more
Abstract (arXiv)

Multimodal large language models (MLLMs) have made significant progress in visual-language reasoning, but their ability to efficiently handle long videos remains limited. Despite recent advances in long-context MLLMs, storing and attending to the key-value (KV) cache for long visual contexts incurs substantial memory and computational overhead. Existing visual compression methods require either encoding the entire visual context before compression or having access to the questions in advance, which is impractical for long video understanding and multi-turn conversational settings. In this work, we propose StreamMem, a query-agnostic KV cache memory mechanism for streaming video understanding. Specifically, StreamMem encodes new video frames in a streaming manner, compressing the KV cache using attention scores between visual tokens and generic query tokens, while maintaining a fixed-size KV memory to enable efficient question answering (QA) in memory-constrained, long-video scenarios. Evaluation on three long video understanding and two streaming video question answering benchmarks shows that StreamMem achieves state-of-the-art performance in query-agnostic KV cache compression and is competitive with query-aware compression approaches.

Key facts. Cited by 58 papers in the map (data through 14 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 57 of its 60 references that are in the map. First posted on arXiv 21 Aug 2025.

Builds on

57 of 60 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025Video-XL-Pro: Reconstructive Token Compression for Extremely Long Video Understanding24 Mar
  2. 2025AdaReTaKe: Adaptive Redundancy Reduction to Perceive Longer for Video-language Understanding16 Mar
  3. 2025Adaptive Keyframe Sampling for Long Video Understanding28 Feb
  4. 2025M-LLM Based Video Frame Selection for Efficient Video Understanding27 Feb
  5. 2024Beyond Training: Dynamic Token Merging for Zero-Shot Video Understanding21 Nov
  6. 2024LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understanding22 Oct
  7. 2024Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding22 Sep
  8. 2024Long Context Transfer from Language to Vision24 Jun
  9. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis31 May
  10. 2024MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understanding8 Apr
  11. 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models28 Nov

Other references · 46

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  3. 2023Visual Instruction Tuningcited by 7,869
  4. 2025Qwen2.5-VL Technical Reportcited by 5,587
  5. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  6. 2024GPT-4o System Cardcited by 4,588
  7. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  8. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  9. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  10. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  11. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  12. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
34 more
  1. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 880
  2. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 832
  3. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
  4. 2024An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Modelscited by 642
  5. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 601
  6. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 583
  7. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 518
  8. 2023Model Tells You What to Discard: Adaptive KV Cache Compression for LLMscited by 474
  9. 2024SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inferencecited by 391
  10. 2023Effective Long-Context Scaling of Foundation Modelscited by 303
  11. 2024MLVU: Benchmarking Multi-task Long Video Understandingcited by 292
  12. 2024VideoLLM-online: Online Video Large Language Model for Streaming Videocited by 225
  13. 2024Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streamscited by 169
  14. 2025LLaVA-Mini: Efficient Image and Video Large Multimodal Models with One Vision Tokencited by 166
  15. 2024Beyond Text-Visual Attention: Exploiting Visual Cues for Effective Token Pruning in VLMscited by 157
  16. 2025Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reactioncited by 127
  17. 2025Streaming Video Question-Answering with In-context Video KV-Cache Retrievalcited by 117
  18. 2024Not All Heads Matter: A Head-Level KV Cache Compression Method with Integrated Retrieval and Reasoningcited by 97
  19. 2024DyCoke: Dynamic Compression of Tokens for Fast Video Large Language Modelscited by 96
  20. 2025KVzip: Query-Agnostic KV Cache Compression with Context Reconstructioncited by 95
  21. 2024A Simple and Effective L2 Norm-Based Strategy for KV Cache Compressioncited by 86
  22. 2024MovieChat+: Question-aware Sparse Memory for Long Video Question Answeringcited by 73
  23. 2024ThinK: Thinner Key Cache by Query-Driven Pruningcited by 68
  24. 2025InfiniPot-V: Memory-Constrained KV Cache Compression for Streaming Video Understandingcited by 58
  25. 2024Koala: Key frame-conditioned long video-LLMcited by 57
  26. 2024Squeezed Attention: Accelerating Long Context Length LLM Inferencecited by 57
  27. 2025LiveVLM: Efficient Online Video Understanding via Streaming-Oriented KV Cache and Retrievalcited by 54
  28. 2025KeyDiff: Key Similarity-Based KV Cache Eviction for Long-Context LLM Inference in Resource-Constrained Environmentscited by 51
  29. 2025Video Compression Commander: Plug-and-Play Inference Acceleration for Video Large Language Modelscited by 38
  30. 2024ReTaKe: Reducing Temporal and Knowledge Redundancy for Long Video Understandingcited by 33
  31. 2024InfiniPot: Infinite Context Processing on Memory-Constrained LLMscited by 27
  32. 2024End-to-End Video Question Answering with Frame Scoring Mechanisms and Adaptive Samplingcited by 19
  33. 2024RankCLIP: Ranking-Consistent Language-Image Pretrainingcited by 9
  34. 2025Beyond RAG: Task-Aware KV Cache Compression for Comprehensive Knowledge Reasoningcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 53.

58 papers.

Sort:

By area:

  1. 2025MemFlow: Flowing Adaptive Memory for Consistent and Efficient Long Video Narrativescited by 45
  2. 2025StreamingTOM: Streaming Token Compression for Efficient Video Understandingcited by 43
  3. 2026HERMES: KV Cache as Hierarchical Memory for Efficient Streaming Video Understandingcited by 41
  4. 2025Accelerating Streaming Video Large Language Models via Hierarchical Token Compressioncited by 37
  5. 2026A Simple Baseline for Streaming Video Understandingcited by 33
  6. 2026FluxMem: Adaptive Hierarchical Memory for Streaming Video Understandingcited by 28
  7. 2026Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneouslycited by 22
  8. 2026StreamReady: Learning What to Answer and When in Long Streaming Videoscited by 19
  9. 2026Thinking in Streaming Videocited by 16
  10. 2025VideoNSA: Native Sparse Attention Scales Video Understandingcited by 11
  11. 2026STAC: Plug-and-Play Spatio-Temporal Aware Cache Compression for Streaming 3D Reconstructioncited by 9
  12. 2026StreamMeCo: Long-Term Agent Memory Compression for Efficient Streaming Video Understandingcited by 9
38 more
  1. 2026Semantic-Aware Adaptive Visual Memory for Streaming Video Understandingcited by 8
  2. 2026What Should a Streaming Video Model Remember?cited by 7
  3. 2026Event-VStream: Event-Driven Real-Time Understanding for Long Video Streamscited by 6
  4. 2026MuKV: Multi-Grained KV Cache Compression for Long Streaming Video Question-Answeringcited by 6
  5. 2026Vista: Scene-Aware Optimization for Streaming Video Question Answering under Post-Hoc Queriescited by 5
  6. 2026CurveStream: Boosting Streaming Video Understanding in MLLMs via Curvature-Aware Hierarchical Visual Memory Managementcited by 5
  7. 2026CodecSight: Leveraging Video Codec Signals for Efficient Streaming VLM Inferencecited by 5
  8. 2026EGOSTREAM: A Diagnostic Benchmark for Streaming Episodic Memory in Egocentric Visioncited by 5
  9. 2026StreamTTT: Reconciling Real-Time Perception and Long-Term Memory in Streaming VLMscited by 5
  10. 2025video-SALMONN S: Memory-Enhanced Streaming Audio-Visual LLMcited by 4
  11. 2026STRIDE: When to Speak Meets Sequence Denoising for Streaming Video Understandingcited by 4
  12. 2026Don't Pause! Every prediction matters in a streaming videocited by 4
  13. 2026Watch, Remember, Reason: Human-View Video Understanding with MLLMscited by 4
  14. 2026OmniMem: Perturbation-aware Memory Compression for Streaming Audio-Visual LLMscited by 4
  15. 2026Towards a Dynamic and Fixed-budget Memory Bank for Efficient Streaming Video Understandingcited by 4
  16. 2026FOLIO: Focused Semantic Memory for Streaming Video Understandingcited by 4
  17. 2026Dynamic Hub-and-Spoke Memory for Streaming Video Understandingcited by 4
  18. 2026WeaveTime: Stream from Earlier Frames into Emergent Memory in VideoLLMscited by 3
  19. 2026FocusGraph: Graph-Structured Frame Selection for Embodied Long Video Question Answeringcited by 3
  20. 2026From Static Inference to Dynamic Interaction: A Survey of Streaming Large Language Modelscited by 3
  21. 2026Kamera: Unified Position-Invariant Multimodal KV Cache for Training-Free Reusecited by 3
  22. 2026Decouple and Cache: KV Cache Construction for Streaming Video Understandingcited by 2
  23. 2026Audio-Visual Intelligence in Large Foundation Modelscited by 2
  24. 2026Homer: Understanding Long-form Videos with Hierarchical Memory and Agentic Reasoningcited by 2
  25. 2026ShallowStream: Index Shallow then Answer Deep for Streaming Video Understandingcited by 2
  26. 2026Going Down Memory Lane: Scaling Tokens for Video Stream Understanding with Dynamic KV-Cache Memorycited by 1
  27. 2026CoRDS: Coreset-based Representative and Diverse Selection for Streaming Video Understandingcited by 1
  28. 2026Linear Scaling Video VLMs for Long Video Understandingcited by 1
  29. 2026FlowNar: Scalable Streaming Narration for Long-Form Videoscited by 1
  30. 2026ViCoStream: Streaming VideoLLMs Can Run Beyond 100 FPS with Stage-Wise Coordinated Inferencecited by 1
  31. 2026How Well Can Your Video Model Remember? Measuring Memory-Budget Trade-offs in Long Video Understandingcited by 1
  32. 2026Think in Sets for Streaming Video Token Compressioncited by 1
  33. 2026StreamFlow: Dynamic Memory Flows for Streaming Video Understandingcited by 1
  34. 2026Beyond Retrieval: Progressive Latent Memory Evolution for Streaming Video Understandingcited by 1
  35. 2026MEMO: Multi-Level Entity-Aware Memory for Streaming Video Understandingcited by 1
  36. 2026EgoEverything: A Benchmark for Human Behavior Inspired Long Context Egocentric Video Understanding in AR Environmentcited by 0
  37. 2026Learning to See What You Need: Gaze Attention for Multimodal Large Language Modelscited by 0
  38. 2026SLVMBench: Skill Learning from Video Memorycited by 0

50 most cited of 58 All 58 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025StreamKV: Streaming Video Question-Answering with Segment-based KV Cache Retrieval and Compression
  2. 2025CacheFlow: Compressive Streaming Memory for Efficient Long-Form Video Understanding
  3. 2026MemoryCard: Topic-Aware Multi-Modal Clue Compression for Long-Video Question Answering
  4. 2024Streaming Long Video Understanding with Large Language Models
  5. 2023Long Video Understanding with Learnable Retrieval in Video-Language Models
  6. 2025SVBench: A Benchmark with Temporal Multi-Turn Dialogues for Streaming Video Understanding
  7. 2026One Token per Highly Selective Frame: Towards Extreme Compression for Long Video Understanding
  8. 2025Flash-VStream: Efficient Real-Time Understanding for Long Video Streams