mlsanityData to 7 Oct 2026

StreamBridge: Turning Your Offline Video Large Language Model into a Proactive Streaming Assistant

arXiv 2505.05467 ↗First posted 8 May 2025cs.CV cs.AI cs.CLcited by 56

Not on the map. Why?

Abstract (arXiv)

We present StreamBridge, a simple yet effective framework that seamlessly transforms offline Video-LLMs into streaming-capable models. It addresses two fundamental challenges in adapting existing models into online scenarios: (1) limited capability for multi-turn real-time understanding, and (2) lack of proactive response mechanisms. Specifically, StreamBridge incorporates (1) a memory buffer combined with a round-decayed compression strategy, supporting long-context multi-turn interactions, and (2) a decoupled, lightweight activation model that can be effortlessly integrated into existing Video-LLMs, enabling continuous proactive responses. To further support StreamBridge, we construct Stream-IT, a large-scale dataset tailored for streaming video understanding, featuring interleaved video-text sequences and diverse instruction formats. Extensive experiments show that StreamBridge significantly improves the streaming understanding capabilities of offline Video-LLMs across various tasks, outperforming even proprietary models such as GPT-4o and Gemini 1.5 Pro. Simultaneously, it achieves competitive or superior performance on standard video understanding benchmarks.

Key facts. Cited by 56 papers in the map (data through 7 Oct 2026). Builds on 74 of its 84 references that are in the map. First posted on arXiv 8 May 2025.

Builds on

74 of 84 references.

References · 74

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2023Visual Instruction Tuningcited by 7,792
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  5. 2024Qwen2.5 Technical Reportcited by 4,713
  6. 2024GPT-4o System Cardcited by 4,467
  7. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  8. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  9. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  10. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  11. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  12. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
38 more
  1. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  2. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  3. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,461
  4. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
  5. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
  6. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  7. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
  8. 2022Planning-oriented Autonomous Drivingcited by 1,068
  9. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 666
  10. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 644
  11. 2017Towards Automatic Learning of Procedures from Web Instructional Videoscited by 616
  12. 2017TALL: Temporal Activity Localization via Language Querycited by 581
  13. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 536
  14. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 502
  15. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 488
  16. 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understandingcited by 399
  17. 2024Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teacherscited by 389
  18. 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 386
  19. 2024TempCompass: Do Video LLMs Really Understand Videos?cited by 352
  20. 2024LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understandingcited by 341
  21. 2019COIN: A Large-scale Dataset for Comprehensive Instructional Video Analysiscited by 316
  22. 2023Perception Test: A Diagnostic Benchmark for Multimodal Video Modelscited by 310
  23. 2024LongVILA: Scaling Long-Context Visual Language Models for Long Videoscited by 303
  24. 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 301
  25. 2023VTimeLLM: Empower LLM to Grasp Video Momentscited by 253
  26. 2024VideoLLM-online: Online Video Large Language Model for Streaming Videocited by 216
  27. 2024VideoChat-Flash: Hierarchical Compression for Long-Context Video Modelingcited by 212
  28. 2024NVILA: Efficient Frontier Visual Language Modelscited by 206
  29. 2021Anticipative Video Transformercited by 197
  30. 2024Streaming Long Video Understanding with Large Language Modelscited by 164
  31. 2024Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streamscited by 160
  32. 2024Oryx MLLM: On-Demand Spatial-Temporal Understanding at Arbitrary Resolutioncited by 157
  33. 2024StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understandingcited by 135
  34. 2025OVO-Bench: How Far is Your Video-LLMs from Real-World Online Video Understanding?cited by 134
  35. 2024Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Inputcited by 132
  36. 2024SlowFast-LLaVA: A Strong Training-Free Baseline for Video Large Language Modelscited by 123
  37. 2025Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reactioncited by 122
  38. 2024VideoGPT+: Integrating Image and Video Encoders for Enhanced Video Understandingcited by 113

50 most cited of 74 All 74 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 50.

56 papers.

Sort:

By area:

  1. 2025StreamingTOM: Streaming Token Compression for Efficient Video Understandingcited by 41
  2. 2026HERMES: KV Cache as Hierarchical Memory for Efficient Streaming Video Understandingcited by 36
  3. 2025StreamAgent: Towards Anticipatory Agents for Streaming Video Understandingcited by 29
  4. 2026A Simple Baseline for Streaming Video Understandingcited by 29
  5. 2026FluxMem: Adaptive Hierarchical Memory for Streaming Video Understandingcited by 26
  6. 2025Streaming Video Instruction Tuningcited by 25
  7. 2025AI for Service: Proactive Assistance with AI Glassescited by 18
  8. 2026StreamReady: Learning What to Answer and When in Long Streaming Videoscited by 18
  9. 2026AURA: Always-On Understanding and Real-Time Assistance via Video Streamscited by 15
  10. 2026Thinking in Streaming Videocited by 14
  11. 2026Semantic-Aware Adaptive Visual Memory for Streaming Video Understandingcited by 8
  12. 2026What Should a Streaming Video Model Remember?cited by 7
38 more
  1. 2026VideoChat3: Fully Open Video MLLM for Efficient and Generalist Video Understandingcited by 6
  2. 2025Building Egocentric Procedural AI Assistant: Methods, Benchmarks, and Challengescited by 5
  3. 2026ROMA: Real-time Omni-Multimodal Assistant with Interactive Streaming Understandingcited by 5
  4. 2026Progressive Online Video Understanding with Evidence-Aligned Timing and Transparent Decisionscited by 5
  5. 2026EvoStreaming: Your Offline Video Model Is a Natively Streaming Assistantcited by 5
  6. 2026Em-Garde: A Propose-Match Framework for Proactive Streaming Video Understandingcited by 4
  7. 2026StreamingClaw Technical Reportcited by 4
  8. 2026From Static Inference to Dynamic Interaction: A Survey of Streaming Large Language Modelscited by 3
  9. 2026StreamPro: From Reactive Perception to Proactive Decision-Making in Streaming Videocited by 3
  10. 2026OmniInteract: Benchmarking Real-World Streaming Interaction for Real-Time Omnimodal Assistantscited by 3
  11. 2026Watch, Remember, Reason: Human-View Video Understanding with MLLMscited by 3
  12. 2026Harnessing Streaming Video in the Wildcited by 3
  13. 2026FOLIO: Focused Semantic Memory for Streaming Video Understandingcited by 3
  14. 2026StreamTTT: Reconciling Real-Time Perception and Long-Term Memory in Streaming VLMscited by 3
  15. 2025Chat with AI: The Surprising Turn of Real-time Video Communication from Human to AIcited by 2
  16. 2026StreamSense: Streaming Social Task Detection with Selective Vision-Language Model Routingcited by 2
  17. 2026WeaveTime: Stream from Earlier Frames into Emergent Memory in VideoLLMscited by 2
  18. 2026STRIDE: When to Speak Meets Sequence Denoising for Streaming Video Understandingcited by 2
  19. 2026Decouple and Cache: KV Cache Construction for Streaming Video Understandingcited by 2
  20. 2026Response-G1: Explicit Scene Graph Modeling for Proactive Streaming Video Understandingcited by 2
  21. 2026Visual Agentic Memory: Enabling Online Long Video Understanding via Online Indexing, Hierarchical Memory, and Agentic Retrievalcited by 2
  22. 2026Omni-DuplexEval: Evaluating Real-time Duplex Omni-modal Interactioncited by 2
  23. 2026Vinci2: Providing Proactive Assistance in Continuous Egocentric Videoscited by 2
  24. 2026ShallowStream: Index Shallow then Answer Deep for Streaming Video Understandingcited by 2
  25. 2026VSAS-Bench: Real-Time Evaluation of Visual Streaming Assistant Modelscited by 1
  26. 2026Don't Pause: Streaming Video-Language Synchrony for Online Video Understandingcited by 1
  27. 2026GuideMe: Multi-Domain Task Guidance and Intervention in Streaming Videocited by 1
  28. 2026GROVE: Growing and Reasoning over Temporally Stratified Memory from Streaming Video Experiencecited by 1
  29. 2026Aero Realtime: Fully Aligned Input-Output Streams for Low-Latency Streaming Multimodal Generationcited by 1
  30. 2026StreamEMS: Streaming Video Understanding with Self-Evolving Memory Scheme for Vision-Language Modelscited by 1
  31. 2026Omni-Streaming Thinkingcited by 1
  32. 2026Think, Act, Build: An Agentic Framework with Vision Language Models for Zero-Shot 3D Visual Groundingcited by 0
  33. 2026Online Reasoning Video Object Segmentationcited by 0
  34. 2026Learning Geometric Representations from Videos for Spatial Intelligent Multimodal Large Language Modelscited by 0
  35. 2026StreamSoccer: Event-Driven Memory for Streaming Soccer Commentarycited by 0
  36. 2026LiveProBench: Can Streaming Video Models Really Interact Like Humans?cited by 0
  37. 2026Live Assistant: Learning Whether, When, and Whom to Assist in Real-World Live Social Streamscited by 0
  38. 2026Agentic Video Understanding: A Surveycited by 0

50 most cited of 56 All 56 (JSON)