# Findings of the Second Shared Task on Multimodal Machine Translation and Multilingual Image Description

arXiv: https://arxiv.org/abs/1710.07177 · HTML: https://mlsanity.com/paper/1710.07177/

## Key facts

- Cited by 76 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Oct 2017.

## Built on by (most cited first, 30 of 76)

- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learning](https://mlsanity.com/paper/2103.01913/) (2021)
- [Crossmodal-3600: A Massively Multilingual Multimodal Evaluation Dataset](https://mlsanity.com/paper/2205.12522/) (2022)
- [UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training](https://mlsanity.com/paper/2104.00332/) (2021)
- [IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languages](https://mlsanity.com/paper/2201.11732/) (2022)
- [LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval](https://mlsanity.com/paper/2103.08784/) (2021)
- [Good for Misconceived Reasons: An Empirical Revisiting on the Need for Visual Context in Multimodal Machine Translation](https://mlsanity.com/paper/2105.14462/) (2021)
- [MURAL: Multimodal, Multitask Retrieval Across Languages](https://mlsanity.com/paper/2109.05125/) (2021)
- [Language Model Alignment with Elastic Reset](https://mlsanity.com/paper/2312.07551/) (2023)
- [On Vision Features in Multimodal Machine Translation](https://mlsanity.com/paper/2203.09173/) (2022)
- [Retrieve Fast, Rerank Smart: Cooperative and Joint Approaches for Improved Cross-Modal Retrieval](https://mlsanity.com/paper/2103.11920/) (2021)
- [Bloom Library: Multimodal Datasets in 300+ Languages for a Variety of Downstream Tasks](https://mlsanity.com/paper/2210.14712/) (2022)
- [VALHALLA: Visual Hallucination for Machine Translation](https://mlsanity.com/paper/2206.00100/) (2022)
- [A Survey on Multi-modal Summarization](https://mlsanity.com/paper/2109.05199/) (2021)
- [Tackling Ambiguity with Images: Improved Multimodal Machine Translation and Contrastive Evaluation](https://mlsanity.com/paper/2212.10140/) (2022)
- [Vision Matters When It Should: Sanity Checking Multimodal Machine Translation Models](https://mlsanity.com/paper/2109.03415/) (2021)
- [LVP-M3: Language-aware Visual Prompt for Multilingual Multimodal Machine Translation](https://mlsanity.com/paper/2210.15461/) (2022)
- [GEM: A General Evaluation Benchmark for Multimodal Tasks](https://mlsanity.com/paper/2106.09889/) (2021)
- [Guiding Visual Question Generation](https://mlsanity.com/paper/2110.08226/) (2021)
- [Distill the Image to Nowhere: Inversion Knowledge Distillation for Multimodal Machine Translation](https://mlsanity.com/paper/2210.04468/) (2022)
- [VISA: An Ambiguous Subtitles Dataset for Visual Scene-Aware Machine Translation](https://mlsanity.com/paper/2201.08054/) (2022)
- [Babel-ImageNet: Massively Multilingual Evaluation of Vision-and-Language Representations](https://mlsanity.com/paper/2306.08658/) (2023)
- [MultiSubs: A Large-scale Multimodal and Multilingual Dataset](https://mlsanity.com/paper/2103.01910/) (2021)
- [Maximum Likelihood Estimation for Multimodal Learning with Missing Modality](https://mlsanity.com/paper/2108.10513/) (2021)
- [Product-oriented Machine Translation with Cross-modal Cross-lingual Pre-training](https://mlsanity.com/paper/2108.11119/) (2021)
- [MaXM: Towards Multilingual Visual Question Answering](https://mlsanity.com/paper/2209.05401/) (2022)
- [BIG-C: a Multimodal Multi-Purpose Dataset for Bemba](https://mlsanity.com/paper/2305.17202/) (2023)
- [Translation-Enhanced Multilingual Text-to-Image Generation](https://mlsanity.com/paper/2305.19216/) (2023)
- [mOSCAR: A Large-scale Multilingual and Multimodal Document-level Corpus](https://mlsanity.com/paper/2406.08707/) (2024)
- [Delving Deeper into Cross-lingual Visual Question Answering](https://mlsanity.com/paper/2202.07630/) (2022)

## Abstract (arXiv)

We present the results from the second shared task on multimodal machine translation and multilingual image description. Nine teams submitted 19 systems to two tasks. The multimodal translation task, in which the source sentence is supplemented by an image, was extended with a new language (French) and two new test sets. The multilingual image description task was changed such that at test time, only the image is given. Compared to last year, multimodal systems improved, but text-only systems remain competitive.