# TRACE: Temporal Audit and Condition-aware Evaluation of Streaming Video Understanding

arXiv: https://arxiv.org/abs/2609.30670 · HTML: https://mlsanity.com/paper/2609.30670/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 31 references that are in the map.
- First posted on arXiv 25 Sep 2026.

## Builds on (most cited first, 29 of 29)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [Video-LLaVA: Learning United Visual Representation by Alignment Before Projection](https://mlsanity.com/paper/2311.10122/) (2023)
- [EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understanding](https://mlsanity.com/paper/2308.09126/) (2023)
- [LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding](https://mlsanity.com/paper/2407.15754/) (2024)
- [LVBench: An Extreme Long Video Understanding Benchmark](https://mlsanity.com/paper/2406.08035/) (2024)
- [MLVU: Benchmarking Multi-task Long Video Understanding](https://mlsanity.com/paper/2406.04264/) (2024)
- [Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Language](https://mlsanity.com/paper/1912.03590/) (2019)
- [VideoLLM-online: Online Video Large Language Model for Streaming Video](https://mlsanity.com/paper/2406.11816/) (2024)
- [StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understanding](https://mlsanity.com/paper/2411.03628/) (2024)
- [OVO-Bench: How Far is Your Video-LLMs from Real-World Online Video Understanding?](https://mlsanity.com/paper/2501.05510/) (2025)
- [Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilities](https://mlsanity.com/paper/2503.04721/) (2025)
- [MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interaction](https://mlsanity.com/paper/2604.27393/) (2026)
- [QVHighlights: Detecting Moments and Highlights in Videos via Natural Language Queries](https://mlsanity.com/paper/2107.09609/) (2021)
- [Online Video Understanding: OVBench and VideoChat-Online](https://mlsanity.com/paper/2501.00584/) (2024)
- [OmniMMI: A Comprehensive Multi-modal Interaction Benchmark in Streaming Video Contexts](https://mlsanity.com/paper/2503.22952/) (2025)
- [Eyes Wide Open: Ego Proactive Video-LLM for Streaming Video](https://mlsanity.com/paper/2510.14560/) (2025)
- [ProactiveVideoQA: A Comprehensive Benchmark Evaluating Proactive Interactions in Video Large Language Models](https://mlsanity.com/paper/2507.09313/) (2025)
- [Thinking in Streaming Video](https://mlsanity.com/paper/2603.12938/) (2026)
- [AURA: Always-On Understanding and Real-Time Assistance via Video Streams](https://mlsanity.com/paper/2604.04184/) (2026)
- [RIVER: A Real-Time Interaction Benchmark for Video LLMs](https://mlsanity.com/paper/2603.03985/) (2026)
- [JoyAI-VL-Interaction: Real-Time Vision-Language Interaction Intelligence](https://mlsanity.com/paper/2606.14777/) (2026)
- [OmniPro: A Comprehensive Benchmark for Omni-Proactive Streaming Video Understanding](https://mlsanity.com/paper/2605.18577/) (2026)
- [IPIBench: Evaluating Interactive Proactive Intelligence of MLLMs under Continuous Streams](https://mlsanity.com/paper/2605.27074/) (2026)
- [MOSS-VL Technical Report](https://mlsanity.com/paper/2608.15045/) (2026)
- [OmniInteract: Benchmarking Real-World Streaming Interaction for Real-Time Omnimodal Assistants](https://mlsanity.com/paper/2605.26485/) (2026)
- [MOSS-Video-Preview: Toward Real-Time Video Understanding via Cross-Attention](https://mlsanity.com/paper/2606.07639/) (2026)
- [S-EMBER: A Large-Scale Benchmark for Streaming Egocentric Memory Retrieval](https://mlsanity.com/paper/2607.02689/) (2026)

## Abstract (arXiv)

Streaming video understanding requires models to interpret evidence as it arrives, yet current evaluations often report task scores without specifying when evidence becomes valid, how visual history is maintained, or how responses are triggered. As a result, similar scores may correspond to different workloads, failure modes, and operational behavior. We introduce TRACE (Temporal Audit and Condition-aware Evaluation), a condition-aware benchmark and evaluation framework that makes these factors explicit. TRACE combines temporally audited visual tasks with evidence timing and instruction-dependent trigger annotations, a unified causal Core--Adapter protocol that controls information availability while recording actual history processing and response events, and multidimensional reporting of answer quality, timeliness, response-selection behavior, workload, completion, and reliability. On 1,240 records from 517 videos, we evaluate eight publicly available models or systems in eight configurations. We find that nearly identical QA accuracy can mask substantial differences in completion, answer validity, and generation workload, while proactive performance separates into response quality, response delay, false alarms (responses emitted while no target window is currently valid and a later one remains), and missed target windows. These results show that streaming-video performance should be interpreted as execution-conditioned system behavior rather than a single score. Our benchmark and code can be accessed at hrefhttps://github.com/om-ai-lab/trace-benchhttps://github.com/om-ai-lab/trace-bench.