# Learning Temporal Evidence Selection for Video Anomaly Understanding

arXiv: https://arxiv.org/abs/2610.16299 · HTML: https://mlsanity.com/paper/2610.16299/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 32 of its 39 references that are in the map.
- First posted on arXiv 12 Oct 2026.

## Builds on (most cited first, 30 of 32)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding](https://mlsanity.com/paper/2306.02858/) (2023)
- [Video-LLaVA: Learning United Visual Representation by Alignment Before Projection](https://mlsanity.com/paper/2311.10122/) (2023)
- [Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models](https://mlsanity.com/paper/2306.05424/) (2023)
- [LLaVA-Video: Video Instruction Tuning With Synthetic Data](https://mlsanity.com/paper/2410.02713/) (2024)
- [Real-world Anomaly Detection in Surveillance Videos](https://mlsanity.com/paper/1801.04264/) (2018)
- [Recognize Anything: A Strong Image Tagging Model](https://mlsanity.com/paper/2306.03514/) (2023)
- [Weakly-supervised Video Anomaly Detection with Robust Temporal Feature Magnitude Learning](https://mlsanity.com/paper/2101.10030/) (2021)
- [NVILA: Efficient Frontier Visual Language Models](https://mlsanity.com/paper/2412.04468/) (2024)
- [Adaptive Keyframe Sampling for Long Video Understanding](https://mlsanity.com/paper/2502.21271/) (2025)
- [MIST: Multiple Instance Self-Training Framework for Video Anomaly Detection](https://mlsanity.com/paper/2104.01633/) (2021)
- [MGFN: Magnitude-Contrastive Glance-and-Focus Network for Weakly-Supervised Video Anomaly Detection](https://mlsanity.com/paper/2211.15098/) (2022)
- [Harnessing Large Language Models for Training-free Video Anomaly Detection](https://mlsanity.com/paper/2404.01014/) (2024)
- [Q-Frame: Query-aware Frame Selection and Multi-Resolution Adaptation for Video-LLMs](https://mlsanity.com/paper/2506.22139/) (2025)
- [Holmes-VAU: Towards Long-term Video Anomaly Understanding at Any Granularity](https://mlsanity.com/paper/2412.06171/) (2024)
- [Follow the Rules: Reasoning for Video Anomaly Detection with Large Language Models](https://mlsanity.com/paper/2407.10299/) (2024)
- [Hawk: Learning to Understand Open-World Video Anomalies](https://mlsanity.com/paper/2405.16886/) (2024)
- [Video Anomaly Detection and Explanation via Large Language Models](https://mlsanity.com/paper/2401.05702/) (2024)
- [Uncovering What, Why and How: A Comprehensive Benchmark for Causation Understanding of Video Anomaly](https://mlsanity.com/paper/2405.00181/) (2024)
- [Generative Frame Sampler for Long Video Understanding](https://mlsanity.com/paper/2503.09146/) (2025)
- [PANDA: Towards Generalist Video Anomaly Detection via Agentic AI Engineer](https://mlsanity.com/paper/2509.26386/) (2025)
- [VADTree: Explainable Training-Free Video Anomaly Detection via Hierarchical Granularity-Aware Tree](https://mlsanity.com/paper/2510.22693/) (2025)
- [Sherlock: Towards Multi-scene Video Abnormal Event Extraction and Localization via a Global-local Spatial-sensitive LLM](https://mlsanity.com/paper/2502.18863/) (2025)
- [VAGU & GtS: LLM-Based Benchmark and Framework for Joint Video Anomaly Grounding and Understanding](https://mlsanity.com/paper/2507.21507/) (2025)
- [Aligning Effective Tokens with Video Anomaly in Large Language Models](https://mlsanity.com/paper/2508.06350/) (2025)
- [B-VLLM: A Vision Large Language Model with Balanced Spatio-Temporal Tokens](https://mlsanity.com/paper/2412.09919/) (2024)
- [CueBench: Advancing Unified Understanding of Context-Aware Video Anomalies in Real-World](https://mlsanity.com/paper/2511.00613/) (2025)

## Abstract (arXiv)

Video anomaly understanding (VAU) aims to generate language-level interpretations of anomalous events in long videos, requiring visual evidence that supports explanation beyond anomaly detection alone. Existing VAU pipelines commonly rely on anomaly-centric or predefined temporal sampling strategies, which efficiently emphasize suspicious moments but overlook contextual cues, object interactions, and temporally distributed evidence needed for reasoning. Motivated by recent progress in learnable temporal selection for long-video question answering, we revisit VAU temporal sampling as a downstream-aligned evidence selection problem. We introduce the semantic-conditioned Temporal Sampling Q-Former that learns to select temporal visual evidence through the VAU generation objective. Since generic VAU prompts provide weak video-specific guidance, we condition the sampler on CLIP-embedded object tags automatically extracted from the input video. These tags are used only for temporal evidence selection and are not appended to the LLM generation prompt, separating sampler conditioning from prompt augmentation. The proposed sampler operates in visual-token space, computes soft temporal selection weights from frame-level representations, and merges duplicated selections before response generation. Experiments on HIVAU-70K show that our model achieves the best performance among the compared fine-tuned models, while ablations demonstrate that learnable temporal sampling improves over fixed anomaly-focused sampling and that CLIP-embedded object-tag conditioning further improves over generic-prompt conditioning. Our results suggest that temporal sampling in VAU should be aligned with explanation-oriented evidence utility rather than anomaly saliency alone.