mlsanityData to 14 Oct 2026

Multi30K: Multilingual English-German Image Descriptions

arXiv 1605.00459 ↗First posted 2 May 2016cs.CL cs.CVcited by 250

Not on the map. Why?

Cited by 29 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce the Multi30K dataset to stimulate multilingual multimodal research. Recent advances in image description have been demonstrated on English-language datasets almost exclusively, but image description should not be limited to English. This dataset extends the Flickr30K dataset with i) German translations created by professional translators over a subset of the English descriptions, and ii) descriptions crowdsourced independently of the original English descriptions. We outline how the data can be used for multilingual image description and multimodal machine translation, but we anticipate the data will be useful for a broader range of tasks.

Key facts. Cited by 250 papers in the map (data through 14 Oct 2026). First posted on arXiv 2 May 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 47; 2022: 48; 2023: 51; 2024: 50; 2025: 40; 2026: 14.

250 papers.

Sort:

By area:

  1. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  2. 2021SimVLM: Simple Visual Language Model Pretraining with Weak Supervisioncited by 647
  3. 2021Unifying Vision-and-Language Tasks via Text Generationcited by 412
  4. 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Modelscited by 311
  5. 2021WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learningcited by 287
  6. 2024AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmarkcited by 140
  7. 2021From Show to Tell: A Survey on Deep Learning-based Image Captioningcited by 138
  8. 2023M3IT: A Large-Scale Dataset towards Multi-Modal Multilingual Instruction Tuningcited by 126
  9. 2022Crossmodal-3600: A Massively Multilingual Multimodal Evaluation Datasetcited by 100
  10. 2022AltCLIP: Altering the Language Encoder in CLIP for Extended Language Capabilitiescited by 85
  11. 2023Retrieving Multimodal Information for Augmented Generation: A Surveycited by 85
  12. 2021xGQA: Cross-Lingual Visual Question Answeringcited by 72
38 more
  1. 2021Contrastive Out-of-Distribution Detection for Pretrained Transformerscited by 71
  2. 2023A Comprehensive Survey on Applications of Transformers for Deep Learning Taskscited by 71
  3. 2021UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-trainingcited by 64
  4. 2021Pufferfish: Communication-efficient Models At No Extra Costcited by 61
  5. 2022IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languagescited by 54
  6. 2021Consensus Control for Decentralized Deep Learningcited by 49
  7. 2021LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrievalcited by 44
  8. 2021Good for Misconceived Reasons: An Empirical Revisiting on the Need for Visual Context in Multimodal Machine Translationcited by 44
  9. 2021Grounding 'Grounding' in NLPcited by 44
  10. 2021MURAL: Multimodal, Multitask Retrieval Across Languagescited by 44
  11. 2025Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generationcited by 42
  12. 2021Multilingual Multimodal Pre-training for Zero-Shot Cross-Lingual Transfer of Vision-Language Modelscited by 41
  13. 2021Approximating How Single Head Attention Learnscited by 38
  14. 2021Assessing Multilingual Fairness in Pre-trained Multimodal Representationscited by 35
  15. 2023Language Model Alignment with Elastic Resetcited by 34
  16. 2021Unsupervised Out-of-Domain Detection via Pre-trained Transformerscited by 31
  17. 2022On Vision Features in Multimodal Machine Translationcited by 31
  18. 2022Visualize Before You Write: Imagination-Guided Open-Ended Text Generationcited by 30
  19. 2023AltDiffusion: A Multilingual Text-to-Image Diffusion Modelcited by 30
  20. 2024Parrot: Multilingual Visual Instruction Tuningcited by 30
  21. 2023Is Fine-tuning Needed? Pre-trained Language Models Are Near Perfect for Out-of-Domain Detectioncited by 29
  22. 2021Retrieve Fast, Rerank Smart: Cooperative and Joint Approaches for Improved Cross-Modal Retrievalcited by 28
  23. 2022Bloom Library: Multimodal Datasets in 300+ Languages for a Variety of Downstream Taskscited by 28
  24. 2022VALHALLA: Visual Hallucination for Machine Translationcited by 27
  25. 2023Scene Graph as Pivoting: Inference-time Image-free Unsupervised Multimodal Machine Translation with Visual Scene Hallucinationcited by 27
  26. 2022Neural Machine Translation with Phrase-Level Universal Visual Representationscited by 26
  27. 2022Tackling Ambiguity with Images: Improved Multimodal Machine Translation and Contrastive Evaluationcited by 26
  28. 2024FoodieQA: A Multimodal Dataset for Fine-Grained Understanding of Chinese Food Culturecited by 25
  29. 2022Cross-View Language Modeling: Towards Unified Cross-Lingual Cross-Modal Pre-trainingcited by 24
  30. 2022How Does Adaptive Optimization Impact Local Neural Network Geometry?cited by 24
  31. 2021Uncertainty-Aware Reliable Text Classificationcited by 23
  32. 2023Exploring Better Text Image Translation with Multimodal Codebookcited by 23
  33. 2022Cross-Lingual Cross-Modal Retrieval with Noise-Robust Learningcited by 22
  34. 2021Vision Matters When It Should: Sanity Checking Multimodal Machine Translation Modelscited by 21
  35. 2022LVP-M3: Language-aware Visual Prompt for Multilingual Multimodal Machine Translationcited by 21
  36. 2024The Evolution of Multimodal Model Architecturescited by 21
  37. 2021Gumbel-Attention for Multi-modal Machine Translationcited by 20
  38. 2024Fairness and Bias in Multimodal AI: A Surveycited by 20

50 most cited of 250 All 250 (JSON)