mlsanityData to 7 Oct 2026

MultiVENT: Multilingual Videos of Events with Aligned Natural Text

arXiv 2307.03153 ↗First posted 6 Jul 2023cs.IR cs.CV cs.MMcited by 7

Not on the map. Why?

Abstract (arXiv)

Everyday news coverage has shifted from traditional broadcasts towards a wide range of presentation formats such as first-hand, unedited video footage. Datasets that reflect the diverse array of multimodal, multilingual news sources available online could be used to teach models to benefit from this shift, but existing news video datasets focus on traditional news broadcasts produced for English-speaking audiences. We address this limitation by constructing MultiVENT, a dataset of multilingual, event-centric videos grounded in text documents across five target languages. MultiVENT includes both news broadcast videos and non-professional event footage, which we use to analyze the state of online news videos and how they can be leveraged to build robust, factually accurate models. Finally, we provide a model for complex, multilingual video retrieval to serve as a baseline for information retrieval using MultiVENT.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 41 of its 58 references that are in the map. First posted on arXiv 6 Jul 2023.

Builds on

41 of 58 references.

References · 41

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  5. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  6. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  7. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  8. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,609
  9. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  10. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  11. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
  12. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
29 more
  1. 2017Localizing Moments in Video with Natural Languagecited by 660
  2. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  3. 2022InternVideo: General Video Foundation Models via Generative and Discriminative Learningcited by 441
  4. 2019VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Researchcited by 412
  5. 2021CLIP2Video: Mastering Video-Text Retrieval via Image CLIPcited by 223
  6. 2022MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Textcited by 216
  7. 2020TVR: A Large-Scale Dataset for Video-Subtitle Moment Retrievalcited by 215
  8. 2021PAQ: 65 Million Probably-Asked Questions and What You Can Do With Themcited by 198
  9. 2022mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connectionscited by 185
  10. 2023mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Videocited by 154
  11. 2021Simple Entity-Centric Questions Challenge Dense Retrieverscited by 140
  12. 2022Bridging Video-text Retrieval with Multiple Choice Questionscited by 79
  13. 2020Cross-media Structured Common Space for Multimedia Event Extractioncited by 59
  14. 2021Object-aware Video-language Pre-training for Retrievalcited by 57
  15. 2022A CLIP-Hitchhiker's Guide to Long Video Retrievalcited by 57
  16. 2021Multilingual Multimodal Pre-training for Zero-Shot Cross-Lingual Transfer of Vision-Language Modelscited by 41
  17. 2023A Large Cross-Modal Video Retrieval Dataset with Reading Comprehensioncited by 31
  18. 2023WebBrain: Learning to Generate Factually Correct Articles for Queries by Grounding on Large Web Corpuscited by 29
  19. 2022Watching the News: Towards VideoQA Models that can Readcited by 20
  20. 2021MuMuQA: Multimedia Multi-Hop News Question Answering via Cross-Media Knowledge Extraction and Groundingcited by 19
  21. 2020TRECVID 2019: An Evaluation Campaign to Benchmark Video Activity Detection, Video Captioning and Matching, and Video Search & Retrievalcited by 16
  22. 2021Narrative Question Answering with Cutting-Edge Open-Domain QA Techniques: A Comprehensive Studycited by 15
  23. 2018V3C - a Research Video Collectioncited by 12
  24. 2022Ambiguous Images With Human Judgments for Robust Visual Event Classificationcited by 12
  25. 2023MAUPQA: Massive Automatically-created Polish Question Answering Datasetcited by 11
  26. 2021MTVR: Multilingual Moment Retrieval in Videoscited by 8
  27. 2023ChinaOpen: A Dataset for Open-world Multimodal Learningcited by 4
  28. 2021Rudder: A Cross Lingual Video and Text Retrieval Datasetcited by 3
  29. 2022C2KD: Cross-Lingual Cross-Modal Knowledge Distillation for Multilingual Text-Video Retrievalcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 2; 2026: 1.

7 papers.

  1. 2024MMToM-QA: Multimodal Theory of Mind Question Answeringcited by 84
  2. 2024MuMA-ToM: Multi-modal Multi-Agent Theory of Mindcited by 51
  3. 2024MultiVENT 2.0: A Massive Multilingual Benchmark for Event-Centric Video Retrievalcited by 17
  4. 2024A Survey of Video Datasets for Grounded Event Understandingcited by 9
  5. 2025MMMORRF: Multimodal Multilingual Modularized Reciprocal Rank Fusioncited by 8
  6. 2025Bonsai: Interpretable Tree-Adaptive Grounded Reasoningcited by 1
  7. 2026When Jokes Cross the Line: Analyzing Regular Humor and Dark Humor in YouTube Shortscited by 0