mlsanityData to 7 Oct 2026

A Visual Attention Grounding Neural Model for Multimodal Machine Translation

arXiv 1808.08266 ↗First posted 24 Aug 2018cs.CLcited by 22

Not on the map. Why?

Abstract (arXiv)

We introduce a novel multimodal machine translation model that utilizes parallel visual and textual information. Our model jointly optimizes the learning of a shared visual-language embedding and a translator. The model leverages a visual attention grounding mechanism that links the visual semantics with the corresponding textual semantics. Our approach achieves competitive state-of-the-art results on the Multi30K and the Ambiguous COCO datasets. We also collected a new multilingual multimodal product description dataset to simulate a real-world international online shopping scenario. On this dataset, our visual attention grounding model outperforms other methods by a large margin.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Aug 2018.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 9; 2022: 8; 2023: 2; 2024: 3.

22 papers.

Sort:

By area:

  1. 2021WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learningcited by 285
  2. 2021UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-trainingcited by 64
  3. 2021Good for Misconceived Reasons: An Empirical Revisiting on the Need for Visual Context in Multimodal Machine Translationcited by 44
  4. 2021Grounding 'Grounding' in NLPcited by 44
  5. 2022VALHALLA: Visual Hallucination for Machine Translationcited by 27
  6. 2022Neural Machine Translation with Phrase-Level Universal Visual Representationscited by 26
  7. 2022LVP-M3: Language-aware Visual Prompt for Multilingual Multimodal Machine Translationcited by 21
  8. 2021Gumbel-Attention for Multi-modal Machine Translationcited by 20
  9. 2021A Survey on Low-Resource Neural Machine Translationcited by 16
  10. 2023CLIPTrans: Transferring Visual Knowledge with Pre-trained Models for Multimodal Machine Translationcited by 14
  11. 2021AOMD: An Analogy-aware Approach to Offensive Meme Detection on Social Mediacited by 13
  12. 2022Distill the Image to Nowhere: Inversion Knowledge Distillation for Multimodal Machine Translationcited by 13
10 more
  1. 2021Product-oriented Machine Translation with Cross-modal Cross-lingual Pre-trainingcited by 10
  2. 2024A Survey on Multi-modal Machine Translation: Tasks, Methods and Challengescited by 6
  3. 2022Low-resource Neural Machine Translation with Cross-modal Alignmentcited by 5
  4. 2024Detecting Concrete Visual Tokens for Multimodal Machine Translationcited by 3
  5. 2022Supervised Visual Attention for Simultaneous Multimodal Machine Translationcited by 2
  6. 2023Impact of Visual Context on Noisy Multimodal NMT: An Empirical Study for English to Indian Languagescited by 2
  7. 2022Multilingual Multimodality: A Taxonomical Survey of Datasets, Techniques, Challenges and Opportunitiescited by 1
  8. 2021Backretrieval: An Image-Pivoted Evaluation Metric for Cross-Lingual Text Representations Without Parallel Corporacited by 0
  9. 2022Utilizing Language-Image Pretraining for Efficient and Robust Bilingual Word Alignmentcited by 0
  10. 2024Enhancing E-commerce Product Title Translation with Retrieval-Augmented Generation and Large Language Modelscited by 0