mlsanityData to 7 Oct 2026

Gumbel-Attention for Multi-modal Machine Translation

arXiv 2103.08862 ↗First posted 16 Mar 2021cs.CLcited by 20

Not on the map. Why?

Abstract (arXiv)

Multi-modal machine translation (MMT) improves translation quality by introducing visual information. However, the existing MMT model ignores the problem that the image will bring information irrelevant to the text, causing much noise to the model and affecting the translation quality. This paper proposes a novel Gumbel-Attention for multi-modal machine translation, which selects the text-related parts of the image features. Specifically, different from the previous attention-based method, we first use a differentiable method to select the image information and automatically remove the useless parts of the image features. Experiments prove that our method retains the image features related to the text, and the remaining parts help the MMT model generates better translations.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 12 of its 29 references that are in the map. First posted on arXiv 16 Mar 2021.

Builds on

12 of 29 references.

References · 12

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  3. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  4. 2017OpenNMT: Open-source Toolkit for Neural Machine Translationcited by 251
  5. 2016Multi30K: Multilingual English-German Image Descriptionscited by 250
  6. 2017Doubly-Attentive Decoder for Multi-modal Neural Machine Translationcited by 46
  7. 2017Incorporating Global Visual Features into Attention-Based Neural Machine Translationcited by 39
  8. 2019Distilling Translations with Visual Awarenesscited by 30
  9. 2018A Visual Attention Grounding Neural Model for Multimodal Machine Translationcited by 22
  10. 2020How Does Selective Mechanism Improve Self-Attention Networks?cited by 18
  11. 2017LIUM-CVC Submissions for WMT17 Multimodal Translation Taskcited by 7
  12. 2019From Words to Sentences: A Progressive Learning Approach for Zero-resource Machine Translation with Visual Pivotscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 8; 2023: 2; 2024: 4; 2025: 4; 2026: 1.

20 papers.

Sort:

By area:

  1. 2024MoMa: Efficient Early-Fusion Pre-training with Mixture of Modality-Aware Expertscited by 61
  2. 2022On Vision Features in Multimodal Machine Translationcited by 31
  3. 2022VALHALLA: Visual Hallucination for Machine Translationcited by 27
  4. 2022Neural Machine Translation with Phrase-Level Universal Visual Representationscited by 26
  5. 2022Tackling Ambiguity with Images: Improved Multimodal Machine Translation and Contrastive Evaluationcited by 26
  6. 2024Dynamic Tuning Towards Parameter and Inference Efficiency for ViT Adaptationcited by 25
  7. 2023CLIPTrans: Transferring Visual Knowledge with Pre-trained Models for Multimodal Machine Translationcited by 14
  8. 2022Hausa Visual Genome: A Dataset for Multi-Modal English to Hausa Machine Translationcited by 13
  9. 2022Distill the Image to Nowhere: Inversion Knowledge Distillation for Multimodal Machine Translationcited by 13
  10. 2022Increasing Visual Awareness in Multimodal Neural Machine Translation from an Information Theoretic Perspectivecited by 10
  11. 2024Decoupling Weighing and Selecting for Integrating Multiple Graph Pre-training Taskscited by 7
  12. 2021Pay Better Attention to Attention: Head Selection in Multilingual and Multi-Domain Sequence Modelingcited by 5
8 more
  1. 2022Low-resource Neural Machine Translation with Cross-modal Alignmentcited by 5
  2. 2024Towards Zero-Shot Multimodal Machine Translationcited by 3
  3. 2023Iterative Adversarial Attack on Image-guided Story Ending Generationcited by 2
  4. 2025Dual-branch Prompting for Multimodal Machine Translationcited by 2
  5. 2025ConECT Dataset: Overcoming Data Scarcity in Context-Aware E-Commerce MTcited by 1
  6. 2025GIIFT: Graph-guided Inductive Image-free Multimodal Machine Translationcited by 1
  7. 2025Empowering Denoising Sequential Recommendation with Large Language Model Embeddingscited by 0
  8. 2026Improving Visual Sensitivity of LLMs on Multimodal Machine Translation with Metric-based Loss Weightingcited by 0