# OmniPro: A Comprehensive Benchmark for Omni-Proactive Streaming Video Understanding

arXiv: https://arxiv.org/abs/2605.18577 · HTML: https://mlsanity.com/paper/2605.18577/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 31 references that are in the map.
- First posted on arXiv 18 May 2026.

## Builds on (most cited first, 12 of 12)

- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency](https://mlsanity.com/paper/2508.18265/) (2025)
- [VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs](https://mlsanity.com/paper/2406.07476/) (2024)
- [Qwen2.5-Omni Technical Report](https://mlsanity.com/paper/2503.20215/) (2025)
- [Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs](https://mlsanity.com/paper/2503.01743/) (2025)
- [Qwen3-Omni Technical Report](https://mlsanity.com/paper/2509.17765/) (2025)
- [MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interaction](https://mlsanity.com/paper/2604.27393/) (2026)
- [video-SALMONN 2: Caption-Enhanced Audio-Visual Large Language Models](https://mlsanity.com/paper/2506.15220/) (2025)
- [StreamAgent: Towards Anticipatory Agents for Streaming Video Understanding](https://mlsanity.com/paper/2508.01875/) (2025)
- [Thinking in Streaming Video](https://mlsanity.com/paper/2603.12938/) (2026)
- [Em-Garde: A Propose-Match Framework for Proactive Streaming Video Understanding](https://mlsanity.com/paper/2603.19054/) (2026)
- [STRIDE: When to Speak Meets Sequence Denoising for Streaming Video Understanding](https://mlsanity.com/paper/2603.27593/) (2026)

## Built on by (most cited first, 11 of 11)

- [Realtime-Venus: A full-duplex interaction system with asynchronous delegation](https://mlsanity.com/paper/2609.13814/) (2026)
- [Omni-Streaming Thinking](https://mlsanity.com/paper/2609.15128/) (2026)
- [Omni Demand Understanding: A Benchmark for Contextual User-Intent Inference in Multimodal Interaction](https://mlsanity.com/paper/2609.21392/) (2026)
- [OmniVChat: Synthesizing, Benchmarking, and Training for Native Audio-Visual Dialogue](https://mlsanity.com/paper/2609.21465/) (2026)
- [TRACE: Temporal Audit and Condition-aware Evaluation of Streaming Video Understanding](https://mlsanity.com/paper/2609.30670/) (2026)
- [Agentic Video Understanding: A Survey](https://mlsanity.com/paper/2609.31713/) (2026)
- [APM-Bench: Benchmarking Cross-session Persistent Memory for Egocentric Streaming Video Assistants](https://mlsanity.com/paper/2609.37559/) (2026)
- [Foresight: planning future perception in streaming VLMs without retraining](https://mlsanity.com/paper/2610.03123/) (2026)
- [InteractionBench: A Real-Time Interaction Benchmark for Streaming Video Systems](https://mlsanity.com/paper/2610.05775/) (2026)
- [Improving Proactive AI Assistance with Hierarchical Procedural Understanding](https://mlsanity.com/paper/2610.06505/) (2026)
- [DeltaStreamer: Towards Human-Like Agentic Streaming Video Understanding](https://mlsanity.com/paper/2610.19849/) (2026)

## Abstract (arXiv)

Omni-proactive streaming video understanding, i.e., autonomously deciding when to speak and what to say from continuous audio-visual streams, is an emerging capability of omni-modal large language models. Existing benchmarks fall short in three key aspects: they rely primarily on visual signals, adopt polling or fixed-timestamp protocols instead of true proactive evaluation, and cover only a limited range of tasks, preventing reliable assessment and differentiation of omni-proactive streaming models. We present OmniPro, the first benchmark to jointly evaluate omni-modal perception, proactive responding, and diverse video understanding tasks. It comprises 2,700 human-verified samples spanning 9 sub-tasks and 3 cognitive levels, covering 6 basic video understanding capabilities. Notably, 84% of samples require audio signals (speech or non-speech), and each sample is annotated with modality-isolation labels to enable fine-grained multimodal analysis. We further introduce a dual-mode evaluation protocol: Probe mode assesses content understanding by querying the model before and after each ground-truth trigger, while Online mode evaluates full proactive ability by requiring models to autonomously decide when to respond in streaming input. Evaluating 11 representative models reveals three key findings: (1) audio provides consistent gains but with highly variable utilization across models, (2) performance degrades significantly over time, indicating limited long-horizon robustness, and (3) non-speech audio perception remains the weakest dimension.