Em-Garde: A Propose-Match Framework for Proactive Streaming Video Understanding
Not on the map. Why?
Abstract (arXiv)
Recent advances in Streaming Video Understanding has enabled a new interaction paradigm where models respond proactively to user queries. Current proactive VideoLLMs rely on per-frame triggering decision making, which suffers from an efficiency-accuracy dilemma. We propose Em-Garde, a novel framework that decouples semantic understanding from streaming perception. At query time, the Instruction-Guided Proposal Parser transforms user queries into structured, perceptually grounded visual proposals; during streaming, a Lightweight Proposal Matching Module performs efficient embedding-based matching to trigger responses. Experiments on StreamingBench and OVO-Bench demonstrate consistent improvements over prior models in proactive response accuracy and efficiency, validating an effective solution for proactive video understanding under strict computational constraints.
Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 45 of its 56 references that are in the map. First posted on arXiv 19 Mar 2026.
Builds on
45 of 56 references.
References · 45
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024GPT-4o System Cardcited by 4,588
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
- 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
- 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
- 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
- 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 518
- 2024Long Context Transfer from Language to Visioncited by 515
33 more
- 2024LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understandingcited by 356
- 2019COIN: A Large-scale Dataset for Comprehensive Instructional Video Analysiscited by 318
- 2024VideoLLM-online: Online Video Large Language Model for Streaming Videocited by 225
- 2024GME: Improving Universal Multimodal Retrieval by Multimodal LLMscited by 170
- 2024Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streamscited by 169
- 2024Streaming Long Video Understanding with Large Language Modelscited by 167
- 2024StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understandingcited by 144
- 2025OVO-Bench: How Far is Your Video-LLMs from Real-World Online Video Understanding?cited by 142
- 2024Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Inputcited by 134
- 2020SoccerNet-v2: A Dataset and Benchmarks for Holistic Understanding of Broadcast Soccer Videoscited by 131
- 2025Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reactioncited by 127
- 2025Streaming Video Question-Answering with In-context Video KV-Cache Retrievalcited by 117
- 2025VLM2Vec-V2: Advancing Multimodal Embedding for Videos, Images, and Visual Documentscited by 116
- 2024Video-CCAM: Enhancing Video-Language Understanding with Causal Cross-Attention Masks for Short and Long Videoscited by 98
- 2025TimeChat-Online: 80% Visual Tokens are Naturally Redundant in Streaming Videoscited by 97
- 2025StreamForest: Efficient Online Video Understanding with Persistent Event Memorycited by 78
- 2025StreamBridge: Turning Your Offline Video Large Language Model into a Proactive Streaming Assistantcited by 63
- 2025LION-FS: Fast & Slow Video-Language Thinker as Online Video Assistantcited by 58
- 2024VideoLLM-MoD: Efficient Video-Language Streaming with Mixture-of-Depths Vision Computationcited by 55
- 2025LiveVLM: Efficient Online Video Understanding via Streaming-Oriented KV Cache and Retrievalcited by 54
- 2024VideoLLM Knows When to Speak: Enhancing Time-Sensitive Video Comprehension with Video-Text Duet Interaction Formatcited by 47
- 2025StreamMind: Unlocking Full Frame Rate Streaming Video Dialogue through Event-Gated Cognitioncited by 46
- 2025ViSpeak: Visual Instruction Feedback in Streaming Videoscited by 42
- 2025Accelerating Streaming Video Large Language Models via Hierarchical Token Compressioncited by 37
- 2025StreamAgent: Towards Anticipatory Agents for Streaming Video Understandingcited by 33
- 2025RzenEmbed: Towards Comprehensive Multimodal Retrievalcited by 29
- 2025Eyes Wide Open: Ego Proactive Video-LLM for Streaming Videocited by 28
- 2025ProactiveVideoQA: A Comprehensive Benchmark Evaluating Proactive Interactions in Video Large Language Modelscited by 27
- 2025MMDuet2: Enhancing Proactive Interaction of Video MLLMs with Multi-Turn Reinforcement Learningcited by 22
- 2025AI for Service: Proactive Assistance with AI Glassescited by 20
- 2025Flashback: Memory-Driven Zero-shot, Real-time Video Anomaly Detectioncited by 12
- 2025StreamEQA: Towards Streaming Video Understanding for Embodied Scenarioscited by 12
- 2025Less Redundancy: Boosting Practicality of Vision Language Model in Walking Assistantscited by 2
Built on by
Citing papers per year: 2026: 7.
7 papers.
- 2026OmniPro: A Comprehensive Benchmark for Omni-Proactive Streaming Video Understandingcited by 11
- 2026VideoChat3: Fully Open Video MLLM for Efficient and Generalist Video Understandingcited by 10
- 2026StreamOV: Streaming Omni-Video Understanding via Evidence-Guided Memory and Response Triggeringcited by 5
- 2026Don't Pause: Streaming Video-Language Synchrony for Online Video Understandingcited by 1
- 2026OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interactioncited by 1
- 2026Improving Proactive AI Assistance with Hierarchical Procedural Understandingcited by 0
- 2026DeltaStreamer: Towards Human-Like Agentic Streaming Video Understandingcited by 0
- 2026DeltaStreamer: Towards Human-Like Agentic Streaming Video Understanding14 Oct 2026
- 2026Improving Proactive AI Assistance with Hierarchical Procedural Understanding5 Oct 2026
- 2026OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interaction1 Oct 2026
- 2026VideoChat3: Fully Open Video MLLM for Efficient and Generalist Video Understanding16 Jul 2026
- 2026Don't Pause: Streaming Video-Language Synchrony for Online Video Understanding5 Jun 2026
- 2026StreamOV: Streaming Omni-Video Understanding via Evidence-Guided Memory and Response Triggering25 May 2026
- 2026OmniPro: A Comprehensive Benchmark for Omni-Proactive Streaming Video Understanding18 May 2026