Response-G1: Explicit Scene Graph Modeling for Proactive Streaming Video Understanding
Not on the map. Why?
Abstract (arXiv)
Proactive streaming video understanding requires Video-LLMs to decide when to respond as a video unfolds, a task where existing methods often fall short due to their implicit, query-agnostic modeling of visual evidence. We introduce Response-G1, a novel framework that establishes explicit, structured alignment between the accumulated video evidence and the query's expected response conditions via scene graphs. The framework operates in three fine-tuning-free stages: (1) online query-guided scene graph generation from streaming clips; (2) memory-based retrieval of the most semantically relevant historical scene graphs; and (3) retrieval-augmented trigger prompting for per-frame "silence/response" decisions. By grounding both evidence and conditions in a shared graph representation, Response-G1 achieves more interpretable and accurate response timing decisions. Experimental results on established benchmarks demonstrate the superiority of our method in both proactive and reactive tasks, validating the advantage of explicit scene graph modeling and retrieval in streaming video understanding.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 31 of its 40 references that are in the map. First posted on arXiv 8 May 2026.
Builds on
31 of 40 references.
References · 31
- 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
- 2023VILA: On Pre-training for Visual Language Modelscited by 686
- 2024Long Context Transfer from Language to Visioncited by 502
- 2024LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understandingcited by 341
- 2025LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Trainingcited by 233
- 2024VideoLLM-online: Online Video Large Language Model for Streaming Videocited by 216
- 2024Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Inputcited by 132
- 2025Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reactioncited by 122
- 2025Streaming Video Question-Answering with In-context Video KV-Cache Retrievalcited by 112
- 2021Spatial-Temporal Transformer for Dynamic Scene Graph Generationcited by 95
19 more
- 2024Video-CCAM: Enhancing Video-Language Understanding with Causal Cross-Attention Masks for Short and Long Videoscited by 95
- 2025TimeChat-Online: 80% Visual Tokens are Naturally Redundant in Streaming Videoscited by 90
- 2025Flash-VStream: Efficient Real-Time Understanding for Long Video Streamscited by 78
- 2025StreamForest: Efficient Online Video Understanding with Persistent Event Memorycited by 71
- 2025Streaming Video Understanding and Multi-round Interaction with Memory-enhanced Knowledgecited by 68
- 2024Online Video Understanding: OVBench and VideoChat-Onlinecited by 58
- 2025LION-FS: Fast & Slow Video-Language Thinker as Online Video Assistantcited by 56
- 2025StreamBridge: Turning Your Offline Video Large Language Model into a Proactive Streaming Assistantcited by 56
- 2024Hawk: Learning to Understand Open-World Video Anomaliescited by 47
- 2020Structured Query-Based Image Retrieval Using Scene Graphscited by 33
- 2023Contrastive Video Question Answering via Video Graph Transformercited by 32
- 2025Eyes Wide Open: Ego Proactive Video-LLM for Streaming Videocited by 25
- 2020Image-to-Image Retrieval by Learning Similarity between Scene Graphscited by 23
- 2023HIG: Hierarchical Interlacement Graph Approach to Scene Graph Generation in Video Understandingcited by 21
- 2025AI for Service: Proactive Assistance with AI Glassescited by 18
- 2024HyperGLM: HyperGraph for Video Scene Graph Generation and Anticipationcited by 17
- 2025Fine-Grained Captioning of Long Videos through Scene Graph Consolidationcited by 7
- 2025Hierarchical Event Memory for Accurate and Low-latency Online Video Temporal Groundingcited by 5
- 2022Enabling Harmonious Human-Machine Interaction with Visual-Context Augmented Dialogue System: A Reviewcited by 2
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026What Should a Streaming Video Model Remember?
- 2026StreamReady: Learning What to Answer and When in Long Streaming Videos
- 2026OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interaction
- 2026Beyond Retrieval: Progressive Latent Memory Evolution for Streaming Video Understanding
- 2026StreamFlow: Dynamic Memory Flows for Streaming Video Understanding
- 2026StreamOV: Streaming Omni-Video Understanding via Evidence-Guided Memory and Response Triggering
- 2025RAVU: Retrieval Augmented Video Understanding with Compositional Reasoning over Graph
- 2026LiveStarPro: Proactive Streaming Video Understanding with Hierarchical Memory for Long-Horizon Streams