# The MeMAD Submission to the WMT18 Multimodal Translation Task

arXiv: https://arxiv.org/abs/1808.10802 · HTML: https://mlsanity.com/paper/1808.10802/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Aug 2018.

## Built on by (most cited first, 22 of 22)

- [Unifying Vision-and-Language Tasks via Text Generation](https://mlsanity.com/paper/2102.02779/) (2021)
- [Good for Misconceived Reasons: An Empirical Revisiting on the Need for Visual Context in Multimodal Machine Translation](https://mlsanity.com/paper/2105.14462/) (2021)
- [On Vision Features in Multimodal Machine Translation](https://mlsanity.com/paper/2203.09173/) (2022)
- [Tackling Ambiguity with Images: Improved Multimodal Machine Translation and Contrastive Evaluation](https://mlsanity.com/paper/2212.10140/) (2022)
- [Cross-lingual Visual Pre-training for Multimodal Machine Translation](https://mlsanity.com/paper/2101.10044/) (2021)
- [Universal Multimodal Representation for Language Understanding](https://mlsanity.com/paper/2301.03344/) (2023)
- [CLIPTrans: Transferring Visual Knowledge with Pre-trained Models for Multimodal Machine Translation](https://mlsanity.com/paper/2308.15226/) (2023)
- [Distill the Image to Nowhere: Inversion Knowledge Distillation for Multimodal Machine Translation](https://mlsanity.com/paper/2210.04468/) (2022)
- [Increasing Visual Awareness in Multimodal Neural Machine Translation from an Information Theoretic Perspective](https://mlsanity.com/paper/2210.08478/) (2022)
- [IndicVisionBench: Benchmarking Cultural and Multilingual Understanding in VLMs](https://mlsanity.com/paper/2511.04727/) (2025)
- [Exploiting Multimodal Reinforcement Learning for Simultaneous Machine Translation](https://mlsanity.com/paper/2102.11387/) (2021)
- [Chitranuvad: Adapting Multi-Lingual LLMs for Multimodal Translation](https://mlsanity.com/paper/2502.20420/) (2025)
- [Multimodal Neural Machine Translation with Search Engine Based Image Retrieval](https://mlsanity.com/paper/2208.00767/) (2022)
- [3AM: An Ambiguity-Aware Multi-Modal Machine Translation Dataset](https://mlsanity.com/paper/2404.18413/) (2024)
- [Incorporating Probing Signals into Multimodal Machine Translation via Visual Question-Answering Pairs](https://mlsanity.com/paper/2310.17133/) (2023)
- [Silo NLP's Participation at WAT2022](https://mlsanity.com/paper/2208.01296/) (2022)
- [RC3: Regularized Contrastive Cross-lingual Cross-modal Pre-training](https://mlsanity.com/paper/2305.07927/) (2023)
- [Impact of Visual Context on Noisy Multimodal NMT: An Empirical Study for English to Indian Languages](https://mlsanity.com/paper/2308.16075/) (2023)
- [Exploring the Necessity of Visual Modality in Multimodal Machine Translation using Authentic Datasets](https://mlsanity.com/paper/2404.06107/) (2024)
- [GIIFT: Graph-guided Inductive Image-free Multimodal Machine Translation](https://mlsanity.com/paper/2507.18562/) (2025)
- [Generalization algorithm of multimodal pre-training model based on graph-text self-supervised training](https://mlsanity.com/paper/2302.10315/) (2023)
- [Modality Influence in Multimodal Machine Learning](https://mlsanity.com/paper/2306.06476/) (2023)

## Abstract (arXiv)

This paper describes the MeMAD project entry to the WMT Multimodal Machine Translation Shared Task. We propose adapting the Transformer neural machine translation (NMT) architecture to a multi-modal setting. In this paper, we also describe the preliminary experiments with text-only translation systems leading us up to this choice. We have the top scoring system for both English-to-German and English-to-French, according to the automatic metrics for flickr18. Our experiments show that the effect of the visual features in our system is small. Our largest gains come from the quality of the underlying text-only NMT system. We find that appropriate use of additional data is effective.