VNU-Bench: A Benchmarking Dataset for Multi-Source Multimodal News Video Understanding
Not on the map. Why?
Abstract (arXiv)
News videos are carefully edited multimodal narratives that combine narration, visuals, and external quotations into coherent storylines. In recent years, there have been significant advances in evaluating multimodal large language models (MLLMs) for news video understanding. However, existing benchmarks largely focus on single-source, intra-video reasoning, where each report is processed in isolation. In contrast, real-world news consumption is inherently multi-sourced: the same event is reported by different outlets with complementary details, distinct narrative choices, and sometimes conflicting claims that unfold over time. Robust news understanding, therefore, requires models to compare perspectives from different sources, align multimodal evidence across sources, and synthesize multi-source information. To fill this gap, we introduce VNU-Bench, the first benchmark for multi-source, cross-video understanding in the news domain. We design a set of new question types that are unique in testing models' ability of understanding multi-source multimodal news from a variety of different angles. We design a novel hybrid human-model QA generation process that addresses the issues of scalability and quality control in building a large dataset for cross-source news understanding. The dataset comprises 429 news groups, 1,405 videos, and 2,501 high-quality questions. Comprehensive evaluation of both closed- and open-source multimodal models shows that VNU-Bench poses substantial challenges for current MLLMs.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 15 of its 34 references that are in the map. First posted on arXiv 6 Jan 2026.
Builds on
15 of 34 references.
References · 15
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
- 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
- 2018TVQA: Localized, Compositional Video Question Answeringcited by 463
- 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 451
- 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 439
- 2024Justice or Prejudice? Quantifying Biases in LLM-as-a-Judgecited by 354
- 2020Visual News: Benchmark and Challenges in News Image Captioningcited by 121
- 2021NewsCLIPpings: Automatic Generation of Out-of-Context Multimodal Mediacited by 106
- 2022End-to-End Multimodal Fact-Checking and Explanation Generation: A Challenging Dataset and Modelscited by 105
- 2022FakeSV: A Multimodal Benchmark with Rich Social Context for Fake News Detection on Short Video Platformscited by 41
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding
- 2025CFVBench: A Comprehensive Video Benchmark for Fine-grained Multimodal Retrieval-Augmented Generation
- 2026Benchmarking Visual State Tracking in Multimodal Video Understanding
- 2026HAVEN: Hierarchically Aligned Multimodal Benchmark for Unified Video Understanding
- 2026Beyond Frame Selection: Rethinking Long-Video Understanding with MLLMs
- 2024ViLBias: Detecting and Reasoning about Bias in Multimodal Content
- 2025CrossVid: A Comprehensive Benchmark for Evaluating Cross-Video Reasoning in Multimodal Large Language Models
- 2024WorldQA: Multimodal World Knowledge in Videos through Long-Chain Reasoning