mlsanityData to 7 Oct 2026

MTVR: Multilingual Moment Retrieval in Videos

arXiv 2108.00061 ↗First posted 30 Jul 2021cs.CL cs.AI cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

We introduce mTVR, a large-scale multilingual video moment retrieval dataset, containing 218K English and Chinese queries from 21.8K TV show video clips. The dataset is collected by extending the popular TVR dataset (in English) with paired Chinese queries and subtitles. Compared to existing moment retrieval datasets, mTVR is multilingual, larger, and comes with diverse annotations. We further propose mXML, a multilingual moment retrieval model that learns and operates on data from both languages, via encoder parameter sharing and language neighborhood constraints. We demonstrate the effectiveness of mXML on the newly collected MTVR dataset, where mXML outperforms strong monolingual baselines while using fewer parameters. In addition, we also provide detailed dataset analyses and model ablations. Data and code are publicly available at https://github.com/jayleicn/mTVRetrieval

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 27 of its 44 references that are in the map. First posted on arXiv 30 Jul 2021.

Builds on

27 of 44 references.

References · 27

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2016Layer Normalizationcited by 5,579
  7. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  8. 2017Dense-Captioning Events in Videoscited by 789
  9. 2017Localizing Moments in Video with Natural Languagecited by 660
  10. 2017TALL: Temporal Activity Localization via Language Querycited by 581
  11. 2018TVQA: Localized, Compositional Video Question Answeringcited by 463
  12. 2019VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Researchcited by 412
15 more
  1. 2020HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-trainingcited by 362
  2. 2020Revisiting Pre-Trained Models for Chinese Natural Language Processingcited by 262
  3. 2016Multi30K: Multilingual English-German Image Descriptionscited by 250
  4. 2020TVR: A Large-Scale Dataset for Video-Subtitle Moment Retrievalcited by 215
  5. 2018A Joint Sequence Fusion Model for Video Question Answering and Retrievalcited by 208
  6. 2017Learning Two-Branch Neural Networks for Image-Text Matching Taskscited by 119
  7. 2018Learning a Text-Video Embedding from Incomplete and Heterogeneous Datacited by 103
  8. 2019ExCL: Extractive Clip Localization Using Natural Language Descriptionscited by 82
  9. 2015Are You Talking to a Machine? Dataset and Methods for Multilingual Image Question Answeringcited by 80
  10. 2018COCO-CN for Cross-Lingual Image Tagging, Captioning and Retrievalcited by 80
  11. 2017Fluency-Guided Cross-Lingual Image Captioningcited by 33
  12. 2018Parameter Sharing Methods for Multilingual Self-Attentional Translation Modelscited by 33
  13. 2019MULE: Multimodal Universal Language Embeddingcited by 22
  14. 2020Learning to Scale Multilingual Representations for Vision-Language Taskscited by 22
  15. 2016Multilingual Visual Sentiment Concept Matchingcited by 1

Built on by

Cited by year

Citing papers per year: 2022: 4; 2023: 3; 2024: 1.

8 papers.

  1. 2022IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languagescited by 54
  2. 2022Temporal Sentence Grounding in Videos: A Survey and Future Directionscited by 37
  3. 2024Position: Measure Dataset Diversity, Don't Just Claim Itcited by 26
  4. 2022Cross-Lingual Cross-Modal Retrieval with Noise-Robust Learningcited by 22
  5. 2023Dual-view Curricular Optimal Transport for Cross-lingual Cross-modal Retrievalcited by 9
  6. 2023MultiVENT: Multilingual Videos of Events with Aligned Natural Textcited by 7
  7. 2022C2KD: Cross-Lingual Cross-Modal Knowledge Distillation for Multilingual Text-Video Retrievalcited by 3
  8. 2023Cross-Modal Retrieval Meets Inference:Improving Zero-Shot Classification with Cross-Modal Retrievalcited by 3