# From Words to Sentences: A Progressive Learning Approach for Zero-resource Machine Translation with Visual Pivots

arXiv: https://arxiv.org/abs/1906.00872 · HTML: https://mlsanity.com/paper/1906.00872/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jun 2019.

## Built on by (most cited first, 6 of 6)

- [Gumbel-Attention for Multi-modal Machine Translation](https://mlsanity.com/paper/2103.08862/) (2021)
- [A Survey on Low-Resource Neural Machine Translation](https://mlsanity.com/paper/2107.04239/) (2021)
- [Product-oriented Machine Translation with Cross-modal Cross-lingual Pre-training](https://mlsanity.com/paper/2108.11119/) (2021)
- [A Survey on Multi-modal Machine Translation: Tasks, Methods and Challenges](https://mlsanity.com/paper/2405.12669/) (2024)
- [EVOQUER: Enhancing Temporal Grounding with Video-Pivoted BackQuery Generation](https://mlsanity.com/paper/2109.04600/) (2021)
- [ConECT Dataset: Overcoming Data Scarcity in Context-Aware E-Commerce MT](https://mlsanity.com/paper/2506.04929/) (2025)

## Abstract (arXiv)

The neural machine translation model has suffered from the lack of large-scale parallel corpora. In contrast, we humans can learn multi-lingual translations even without parallel texts by referring our languages to the external world. To mimic such human learning behavior, we employ images as pivots to enable zero-resource translation learning. However, a picture tells a thousand words, which makes multi-lingual sentences pivoted by the same image noisy as mutual translations and thus hinders the translation model learning. In this work, we propose a progressive learning approach for image-pivoted zero-resource machine translation. Since words are less diverse when grounded in the image, we first learn word-level translation with image pivots, and then progress to learn the sentence-level translation by utilizing the learned word translation to suppress noises in image-pivoted multi-lingual sentences. Experimental results on two widely used image-pivot translation datasets, IAPR-TC12 and Multi30k, show that the proposed approach significantly outperforms other state-of-the-art methods.