mlsanityData to 7 Oct 2026

StreamingTOM: Streaming Token Compression for Efficient Video Understanding

arXiv 2510.18269 ↗First posted 21 Oct 2025cs.CV cs.AIcited by 41

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

also in 1 more
Abstract (arXiv)

Unlike offline processing, streaming video vision-language models face two fundamental constraints: causality and accumulation. Causality prevents access to future frames that offline methods exploit, while accumulation causes tokens to grow unbounded, creating efficiency bottlenecks. However, existing approaches only regulate post-LLM kv-cache, leaving costly pre-LLM prefill unchanged. We introduce StreamingTOM, a training-free, plug-and-play two-stage framework that addresses both pre-LLM and post-LLM bottlenecks. Causal Temporal Reduction imposes a fixed per-frame budget and selects tokens based on adjacent-frame changes and token saliency, drastically reducing per-frame prefill cost by processing only a compact subset of visual tokens, ensuring predictable latency. Online Quantized Memory stores tokens in 4-bit format, retrieves relevant groups on demand, and dequantizes them, keeping the active kv-cache bounded regardless of stream length. Experiments demonstrate our method achieves 15.7× kv-cache compression ratio; compared to prior SOTA (LiveVLM), it delivers 1.2× lower peak memory and 2× faster TTFT. StreamingTOM achieves state-of-the-art accuracy among training-free methods with an average of 63.8% on offline benchmarks and 55.8% accuracy and 3.7 score on RVS. These results demonstrate that real-time streaming video understanding with bounded active memory is achievable without model retraining.

Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 61 of its 72 references that are in the map. First posted on arXiv 21 Oct 2025.

Builds on

61 of 72 references.

Closest ancestors · 10

Same direction, newest first

  1. 2025Variation-aware Vision Token Dropping for Faster Large Vision-Language Models1 Sep
  2. 2025Revisiting MLLM Token Technology through the Lens of Classical Visual Coding19 Aug
  3. 2025A Survey of Token Compression for Efficient Multimodal Large Language Models27 Jul
  4. 2025TopV: Compatible Token Pruning with Inference Time Optimization for Fast and Low-Memory Multimodal Vision Language Model24 Mar
  5. 2025Global Compression Commander: Plug-and-Play Inference Acceleration for High-Resolution Large Vision-Language Models9 Jan
  6. 2024VisionZip: Longer is Better but Not Necessary in Vision Language Models5 Dec
  7. 2024PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reduction22 Oct
  8. 2024LOOK-M: Look-Once Optimization in KV Cache for Efficient Multimodal Long-Context Inference26 Jun
  9. 2024LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Models22 Mar
  10. 2024An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models11 Mar

Other references · 51

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  4. 2025Qwen2.5-VL Technical Reportcited by 5,418
  5. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  6. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  7. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  8. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  9. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  10. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  11. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
  12. 2022Token Merging: Your ViT But Fastercited by 995
38 more
  1. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 666
  2. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 644
  3. 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 643
  4. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 550
  5. 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 344
  6. 2024LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understandingcited by 341
  7. 2024MLVU: Benchmarking Multi-task Long Video Understandingcited by 275
  8. 2024VideoLLM-online: Online Video Large Language Model for Streaming Videocited by 216
  9. 2024Streaming Long Video Understanding with Large Language Modelscited by 164
  10. 2024Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streamscited by 160
  11. 2024StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understandingcited by 135
  12. 2025Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reactioncited by 122
  13. 2025Streaming Video Question-Answering with In-context Video KV-Cache Retrievalcited by 112
  14. 2025StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modelingcited by 107
  15. 2024Efficient Multimodal Large Language Models: A Surveycited by 95
  16. 2024DyCoke: Dynamic Compression of Tokens for Fast Video Large Language Modelscited by 93
  17. 2025TimeChat-Online: 80% Visual Tokens are Naturally Redundant in Streaming Videoscited by 90
  18. 2025StreamingVLM: Real-Time Understanding for Infinite Video Streamscited by 89
  19. 2025Flash-VStream: Efficient Real-Time Understanding for Long Video Streamscited by 78
  20. 2024MovieChat+: Question-aware Sparse Memory for Long Video Question Answeringcited by 73
  21. 2025Streaming Video Understanding and Multi-round Interaction with Memory-enhanced Knowledgecited by 68
  22. 2024PruneVid: Visual Token Pruning for Efficient Video Large Language Modelscited by 64
  23. 2025HoliTom: Holistic Token Merging for Fast Video Large Language Modelscited by 58
  24. 2025StreamBridge: Turning Your Offline Video Large Language Model into a Proactive Streaming Assistantcited by 56
  25. 2025InfiniPot-V: Memory-Constrained KV Cache Compression for Streaming Video Understandingcited by 54
  26. 2025StreamMem: Query-Agnostic KV Cache Memory for Streaming Video Understandingcited by 53
  27. 2025LiveVLM: Efficient Online Video Understanding via Streaming-Oriented KV Cache and Retrievalcited by 50
  28. 2023TESTA: Temporal-Spatial Token Aggregation for Long-form Video-Language Understandingcited by 49
  29. 2025FastVID: Dynamic Density Pruning for Fast Video Large Language Modelscited by 47
  30. 2025StreamMind: Unlocking Full Frame Rate Streaming Video Dialogue through Event-Gated Cognitioncited by 44
  31. 2025ViSpeak: Visual Instruction Feedback in Streaming Videoscited by 38
  32. 2025Video Compression Commander: Plug-and-Play Inference Acceleration for Video Large Language Modelscited by 38
  33. 2025RewardMap: Tackling Sparse Rewards in Fine-grained Visual Reasoning via Multi-Stage Reinforcement Learningcited by 20
  34. 2025OBS-Diff: Accurate Pruning For Diffusion Models in One-Shotcited by 19
  35. 2025Plug-and-Play 1.x-Bit KV Cache Quantization for Video Large Language Modelscited by 12
  36. 2024Cross-Self KV Cache Pruning for Efficient Vision-Language Inferencecited by 11
  37. 2025ToTRL: Unlock LLM Tree-of-Thoughts Reasoning Potential through Puzzles Solvingcited by 8
  38. 2025TVC: Tokenized Video Compression with Ultra-Low Bit Ratecited by 7

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 38.

41 papers.

Sort:

By area:

  1. 2026HERMES: KV Cache as Hierarchical Memory for Efficient Streaming Video Understandingcited by 36
  2. 2026A Simple Baseline for Streaming Video Understandingcited by 29
  3. 2026LVOmniBench: Pioneering Long Audio-Video Understanding Evaluation for Omnimodal LLMscited by 22
  4. 2025OmniZip: Audio-Guided Dynamic Token Compression for Fast Omnimodal Large Language Modelscited by 21
  5. 2026Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospectscited by 8
  6. 2025MergeMix: A Unified Augmentation Paradigm for Visual and Multi-Modal Understandingcited by 7
  7. 2026Think While Watching: Online Streaming Segment-Level Memory for Multi-Turn Video Reasoning in Multimodal Large Language Modelscited by 7
  8. 2026What Should a Streaming Video Model Remember?cited by 7
  9. 2026DASH: Dynamic Audio-Driven Semantic Chunking for Efficient Omnimodal Token Compressioncited by 6
  10. 2026MuKV: Multi-Grained KV Cache Compression for Long Streaming Video Question-Answeringcited by 6
  11. 2026CurveStream: Boosting Streaming Video Understanding in MLLMs via Curvature-Aware Hierarchical Visual Memory Managementcited by 5
  12. 2026StreamingClaw Technical Reportcited by 4
29 more
  1. 2026OVO-S-Bench: A Hierarchical Benchmark for Streaming Spatial Intelligence in Multimodal LLMscited by 4
  2. 2026Towards a Dynamic and Fixed-budget Memory Bank for Efficient Streaming Video Understandingcited by 4
  3. 2026ObjectStream: Latent Objects as Memory Anchors for Streaming Video Understandingcited by 4
  4. 2026From Static Inference to Dynamic Interaction: A Survey of Streaming Large Language Modelscited by 3
  5. 2026CodecSight: Leveraging Video Codec Signals for Efficient Streaming VLM Inferencecited by 3
  6. 2026Keep What Audio Cannot Say: Context-Preserving Token Pruning for Omni-LLMscited by 3
  7. 2026EGOSTREAM: A Diagnostic Benchmark for Streaming Episodic Memory in Egocentric Visioncited by 3
  8. 2026Watch, Remember, Reason: Human-View Video Understanding with MLLMscited by 3
  9. 2026OmniMem: Perturbation-aware Memory Compression for Streaming Audio-Visual LLMscited by 3
  10. 2026FOLIO: Focused Semantic Memory for Streaming Video Understandingcited by 3
  11. 2026StreamTTT: Reconciling Real-Time Perception and Long-Term Memory in Streaming VLMscited by 3
  12. 2026OmniRefine: Alignment-Aware Cooperative Compression for Efficient Omnimodal Large Language Modelscited by 2
  13. 2026HARD-KV: Head-Adaptive Regularization for Decoding-time KV Compressioncited by 2
  14. 2026ShallowStream: Index Shallow then Answer Deep for Streaming Video Understandingcited by 2
  15. 2025Efficient-VLN: A Simple yet Strong Baseline for Efficient Vision-Language Navigationcited by 1
  16. 2026Homer: Understanding Long-form Videos with Hierarchical Memory and Agentic Reasoningcited by 1
  17. 2026Think in Sets for Streaming Video Token Compressioncited by 1
  18. 2026StreamEMS: Streaming Video Understanding with Self-Evolving Memory Scheme for Vision-Language Modelscited by 1
  19. 2026Omni-Streaming Thinkingcited by 1
  20. 2026Going Down Memory Lane: Scaling Tokens for Video Stream Understanding with Dynamic KV-Cache Memorycited by 0
  21. 2026ViCoStream: Streaming VideoLLMs Can Run Beyond 100 FPS with Stage-Wise Coordinated Inferencecited by 0
  22. 2026SLVMBench: Skill Learning from Video Memorycited by 0
  23. 2026SkyVLaM: Multimodal Large Language Model for UAV Video Understanding in Remote Sensingcited by 0
  24. 2026StreamFlow: Dynamic Memory Flows for Streaming Video Understandingcited by 0
  25. 2026Why Is Video Still So Expensive? A Survey of Inference-Efficiency Mechanisms in Video and Audiovisual LLMscited by 0
  26. 2026DeltaS: Reading the Gated Linear Attention State for KV Cache Eviction in Streaming Videocited by 0
  27. 2026KeyRec: Bounded Visual Memory for Streaming and Long-Video Understandingcited by 0
  28. 2026When to Retrieve, When to Stay: Uncertainty-Aware Temporal Evidence Allocation for Streaming Video-LLMscited by 0
  29. 2026APM-Bench: Benchmarking Cross-session Persistent Memory for Egocentric Streaming Video Assistantscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026EarlyTom: Early Token Compression Completes Fast Video Understanding
  2. 2026VETO: Video Efficient Token Optimization for Vision Language Models
  3. 2025Accelerating Streaming Video Large Language Models via Hierarchical Token Compression
  4. 2026CRAFT: Compression via Recursive Adaptive Fusion of Video Tokens for Vision-Language Models
  5. 2026TTF: Temporal Token Fusion for Efficient Video-Language Model
  6. 2026FluxMem: Adaptive Hierarchical Memory for Streaming Video Understanding
  7. 2026One Token per Highly Selective Frame: Towards Extreme Compression for Long Video Understanding
  8. 2024PVC: Progressive Visual Token Compression for Unified Image and Video Processing in Large Vision-Language Models