mlsanityData to 7 Oct 2026

Multi-modal News Understanding with Professionally Labelled Videos (ReutersViLNews)

arXiv 2401.12419 ↗First posted 23 Jan 2024cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

While progress has been made in the domain of video-language understanding, current state-of-the-art algorithms are still limited in their ability to understand videos at high levels of abstraction, such as news-oriented videos. Alternatively, humans easily amalgamate information from video and language to infer information beyond what is visually observable in the pixels. An example of this is watching a news story, where the context of the event can play as big of a role in understanding the story as the event itself. Towards a solution for designing this ability in algorithms, we present a large-scale analysis on an in-house dataset collected by the Reuters News Agency, called Reuters Video-Language News (ReutersViLNews) dataset which focuses on high-level video-language understanding with an emphasis on long-form news. The ReutersViLNews Dataset consists of long-form news videos collected and labeled by news industry professionals over several years and contains prominent news reporting from around the world. Each video involves a single story and contains action shots of the actual event, interviews with people associated with the event, footage from nearby areas, and more. ReutersViLNews dataset contains videos from seven subject categories: disaster, finance, entertainment, health, politics, sports, and miscellaneous with annotations from high-level to low-level, title caption, visual video description, high-level story description, keywords, and location. We first present an analysis of the dataset statistics of ReutersViLNews compared to previous datasets. Then we benchmark state-of-the-art approaches for four different video-language tasks. The results suggest that news-oriented videos are a substantial challenge for current video-language understanding algorithms and we conclude by providing future directions in designing approaches to solve the ReutersViLNews dataset.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 36 of its 53 references that are in the map. First posted on arXiv 23 Jan 2024.

Builds on

36 of 53 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  5. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  6. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
  7. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
  8. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
  9. 2017Dense-Captioning Events in Videoscited by 789
  10. 2021CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrievalcited by 645
  11. 2017Towards Automatic Learning of Procedures from Web Instructional Videoscited by 616
  12. 2016Hollywood in Homes: Crowdsourcing Data Collection for Activity Understandingcited by 535
24 more
  1. 2021Less is More: ClipBERT for Video-and-Language Learning via Sparse Samplingcited by 484
  2. 2016YouTube-8M: A Large-Scale Video Classification Benchmarkcited by 324
  3. 2022ActionFormer: Localizing Moments of Actions with Transformerscited by 308
  4. 2021CLIP2Video: Mastering Video-Text Retrieval via Image CLIPcited by 223
  5. 2018Moments in Time Dataset: one million videos for event understandingcited by 215
  6. 2018A Joint Sequence Fusion Model for Video Question Answering and Retrievalcited by 208
  7. 2015A Dataset for Movie Descriptioncited by 195
  8. 2020Fine-grained Video-Text Retrieval with Hierarchical Graph Reasoningcited by 180
  9. 2015Sequence to Sequence -- Video to Textcited by 157
  10. 2022End-to-end Generative Pretraining for Multimodal Video Captioningcited by 115
  11. 2020Multi-modal Dense Video Captioningcited by 113
  12. 2022Long Movie Clip Classification with State-Space Video Modelscited by 106
  13. 2022TS2-Net: Token Shift and Selection Transformer for Text-Video Retrievalcited by 104
  14. 2017Towards a Visual Privacy Advisor: Understanding and Predicting Privacy Risks in Imagescited by 101
  15. 2015Describing Videos by Exploiting Temporal Structurecited by 88
  16. 2022Temporal Alignment Networks for Long-term Videocited by 84
  17. 2021MDMMT: Multidomain Multimodal Transformer for Video Retrievalcited by 78
  18. 2022Disentangled Representation Learning for Text-Video Retrievalcited by 77
  19. 2020A Better Use of Audio-Visual Cues: Dense Video Captioning with Bi-modal Transformercited by 75
  20. 2022A CLIP-Hitchhiker's Guide to Long Video Retrievalcited by 57
  21. 2018Move Forward and Tell: A Progressive Generator of Video Descriptionscited by 35
  22. 2021Towards Diverse Paragraph Captioning for Untrimmed Videoscited by 22
  23. 2016Title Generation for User Generated Videoscited by 17
  24. 2023Implicit and Explicit Commonsense for Multi-sentence Video Captioningcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Prompt2LVideos: Exploring Prompts for Understanding Long-Form Multimodal Videoscited by 0
  2. 2026VNU-Bench: A Benchmarking Dataset for Multi-Source Multimodal News Video Understandingcited by 0