mlsanityData to 7 Oct 2026

Multimodal Attention for Neural Machine Translation

arXiv 1609.03976 ↗First posted 13 Sep 2016cs.CL cs.NEcited by 19

Not on the map. Why?

Abstract (arXiv)

The attention mechanism is an important part of the neural machine translation (NMT) where it was reported to produce richer source representation compared to fixed-length encoding sequence-to-sequence models. Recently, the effectiveness of attention has also been explored in the context of image captioning. In this work, we assess the feasibility of a multimodal attention mechanism that simultaneously focus over an image and its natural language description for generating a description in another language. We train several variants of our proposed attention mechanism on the Multi30k multilingual image captioning dataset. We show that a dedicated attention for each modality achieves up to 1.6 points in BLEU and METEOR compared to a textual NMT baseline.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Sep 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 6; 2023: 3; 2024: 5; 2025: 2; 2026: 1.

19 papers.

Sort:

By area:

  1. 2022Learning Multi-View Aggregation In the Wild for Large-Scale 3D Semantic Segmentationcited by 47
  2. 2022On Vision Features in Multimodal Machine Translationcited by 31
  3. 2022Neural Machine Translation with Phrase-Level Universal Visual Representationscited by 26
  4. 2023Exploring Better Text Image Translation with Multimodal Codebookcited by 23
  5. 2021Product-oriented Machine Translation with Cross-modal Cross-lingual Pre-trainingcited by 10
  6. 2024AnyTrans: Translate AnyText in the Image with Large Scale Modelscited by 8
  7. 2024A Survey on Multi-modal Machine Translation: Tasks, Methods and Challengescited by 6
  8. 2025Chitranuvad: Adapting Multi-Lingual LLMs for Multimodal Translationcited by 6
  9. 2022Low-resource Neural Machine Translation with Cross-modal Alignmentcited by 5
  10. 2022Multimodal Neural Machine Translation with Search Engine Based Image Retrievalcited by 4
  11. 20243AM: An Ambiguity-Aware Multi-Modal Machine Translation Datasetcited by 4
  12. 2024DAT: Dialogue-Aware Transformer with Modality-Group Fusion for Human Engagement Estimationcited by 4
7 more
  1. 2023Incorporating Probing Signals into Multimodal Machine Translation via Visual Question-Answering Pairscited by 3
  2. 2021Visual Cues and Error Correction for Translation Robustnesscited by 2
  3. 2022A Multi-stage deep architecture for summary generation of soccer videoscited by 1
  4. 2024Exploring the Necessity of Visual Modality in Multimodal Machine Translation using Authentic Datasetscited by 1
  5. 2025ConECT Dataset: Overcoming Data Scarcity in Context-Aware E-Commerce MTcited by 1
  6. 2023Generalization algorithm of multimodal pre-training model based on graph-text self-supervised trainingcited by 0
  7. 2026Video-guided Machine Translation with Global Video Contextcited by 0