CUNI System for the WMT18 Multimodal Translation Task
Not on the map. Why?
Abstract (arXiv)
We present our submission to the WMT18 Multimodal Translation Task. The main feature of our submission is applying a self-attentive network instead of a recurrent neural network. We evaluate two methods of incorporating the visual features in the model: first, we include the image representation as another input to the network; second, we train the model to predict the visual features and use it as an auxiliary objective. For our submission, we acquired both textual and multimodal additional data. Both of the proposed methods yield significant improvements over recurrent networks and self-attentive textual baselines.
Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). First posted on arXiv 12 Nov 2018.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 5; 2022: 6; 2023: 1; 2024: 0; 2025: 1.
13 papers.
Sort:
- 2021Good for Misconceived Reasons: An Empirical Revisiting on the Need for Visual Context in Multimodal Machine Translationcited by 44
- 2022Neural Machine Translation with Phrase-Level Universal Visual Representationscited by 26
- 2022Tackling Ambiguity with Images: Improved Multimodal Machine Translation and Contrastive Evaluationcited by 26
- 2021Searchable Hidden Intermediates for End-to-End Models of Decomposable Sequence Taskscited by 24
- 2021Vision Matters When It Should: Sanity Checking Multimodal Machine Translation Modelscited by 21
- 2022LVP-M3: Language-aware Visual Prompt for Multilingual Multimodal Machine Translationcited by 21
- 2021Cross-lingual Visual Pre-training for Multimodal Machine Translationcited by 19
- 2022Distill the Image to Nowhere: Inversion Knowledge Distillation for Multimodal Machine Translationcited by 13
- 2022Increasing Visual Awareness in Multimodal Neural Machine Translation from an Information Theoretic Perspectivecited by 10
- 2021The JDDC 2.0 Corpus: A Large-Scale Multimodal Multi-Turn Chinese Dialogue Dataset for E-commerce Customer Servicecited by 5
- 2022Low-resource Neural Machine Translation with Cross-modal Alignmentcited by 5
- 2023Semi-Autoregressive Streaming ASR With Label Contextcited by 5
1 more
- 2025Dual-branch Prompting for Multimodal Machine Translationcited by 2
- 2025Dual-branch Prompting for Multimodal Machine Translation23 Jul 2025
- 2023Semi-Autoregressive Streaming ASR With Label Context19 Sep 2023
- 2022Tackling Ambiguity with Images: Improved Multimodal Machine Translation and Contrastive Evaluation20 Dec 2022
- 2022LVP-M3: Language-aware Visual Prompt for Multilingual Multimodal Machine Translation19 Oct 2022
- 2022Increasing Visual Awareness in Multimodal Neural Machine Translation from an Information Theoretic Perspective16 Oct 2022
- 2022Low-resource Neural Machine Translation with Cross-modal Alignment13 Oct 2022
- 2022Distill the Image to Nowhere: Inversion Knowledge Distillation for Multimodal Machine Translation10 Oct 2022
- 2022Neural Machine Translation with Phrase-Level Universal Visual Representations19 Mar 2022
- 2021The JDDC 2.0 Corpus: A Large-Scale Multimodal Multi-Turn Chinese Dialogue Dataset for E-commerce Customer Service27 Sep 2021
- 2021Vision Matters When It Should: Sanity Checking Multimodal Machine Translation Models8 Sep 2021
- 2021Good for Misconceived Reasons: An Empirical Revisiting on the Need for Visual Context in Multimodal Machine Translation30 May 2021
- 2021Searchable Hidden Intermediates for End-to-End Models of Decomposable Sequence Tasks2 May 2021
1 more
- 2021Good for Misconceived Reasons: An Empirical Revisiting on the Need for Visual Context in Multimodal Machine Translationcited by 44
- 2022Neural Machine Translation with Phrase-Level Universal Visual Representationscited by 26
- 2022Tackling Ambiguity with Images: Improved Multimodal Machine Translation and Contrastive Evaluationcited by 26
- 2021Searchable Hidden Intermediates for End-to-End Models of Decomposable Sequence Taskscited by 24
- 2021Vision Matters When It Should: Sanity Checking Multimodal Machine Translation Modelscited by 21
- 2022LVP-M3: Language-aware Visual Prompt for Multilingual Multimodal Machine Translationcited by 21
- 2021Cross-lingual Visual Pre-training for Multimodal Machine Translationcited by 19
- 2022Distill the Image to Nowhere: Inversion Knowledge Distillation for Multimodal Machine Translationcited by 13
- 2022Increasing Visual Awareness in Multimodal Neural Machine Translation from an Information Theoretic Perspectivecited by 10
- 2021The JDDC 2.0 Corpus: A Large-Scale Multimodal Multi-Turn Chinese Dialogue Dataset for E-commerce Customer Servicecited by 5
- 2022Low-resource Neural Machine Translation with Cross-modal Alignmentcited by 5
- 2023Semi-Autoregressive Streaming ASR With Label Contextcited by 5
12 most cited of 13 All 13 (JSON)
- 2025Dual-branch Prompting for Multimodal Machine Translation23 Jul 2025
- 2023Semi-Autoregressive Streaming ASR With Label Context19 Sep 2023
- 2022Tackling Ambiguity with Images: Improved Multimodal Machine Translation and Contrastive Evaluation20 Dec 2022
- 2022LVP-M3: Language-aware Visual Prompt for Multilingual Multimodal Machine Translation19 Oct 2022
- 2022Increasing Visual Awareness in Multimodal Neural Machine Translation from an Information Theoretic Perspective16 Oct 2022
- 2022Low-resource Neural Machine Translation with Cross-modal Alignment13 Oct 2022
- 2022Distill the Image to Nowhere: Inversion Knowledge Distillation for Multimodal Machine Translation10 Oct 2022
- 2022Neural Machine Translation with Phrase-Level Universal Visual Representations19 Mar 2022
- 2021The JDDC 2.0 Corpus: A Large-Scale Multimodal Multi-Turn Chinese Dialogue Dataset for E-commerce Customer Service27 Sep 2021
- 2021Vision Matters When It Should: Sanity Checking Multimodal Machine Translation Models8 Sep 2021
- 2021Good for Misconceived Reasons: An Empirical Revisiting on the Need for Visual Context in Multimodal Machine Translation30 May 2021
- 2021Searchable Hidden Intermediates for End-to-End Models of Decomposable Sequence Tasks2 May 2021
12 newest of 13 All 13 (JSON)