# IITP at WAT 2021: System description for English-Hindi Multimodal Translation Task

arXiv: https://arxiv.org/abs/2107.01656 · HTML: https://mlsanity.com/paper/2107.01656/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 22 references that are in the map.
- First posted on arXiv 4 Jul 2021.

## Builds on (most cited first, 14 of 14)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [VisualBERT: A Simple and Performant Baseline for Vision and Language](https://mlsanity.com/paper/1908.03557/) (2019)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [OpenNMT: Open-source Toolkit for Neural Machine Translation](https://mlsanity.com/paper/1709.03815/) (2017)
- [UNITER: UNiversal Image-TExt Representation Learning](https://mlsanity.com/paper/1909.11740/) (2019)
- [Exploiting Cross-Sentence Context for Neural Machine Translation](https://mlsanity.com/paper/1704.04347/) (2017)
- [Doubly-Attentive Decoder for Multi-modal Neural Machine Translation](https://mlsanity.com/paper/1702.01287/) (2017)
- [Incorporating Global Visual Features into Attention-Based Neural Machine Translation](https://mlsanity.com/paper/1701.06521/) (2017)
- [Dynamic Context-guided Capsule Network for Multimodal Machine Translation](https://mlsanity.com/paper/2009.02016/) (2020)
- [Hindi Visual Genome: A Dataset for Multimodal English-to-Hindi Machine Translation](https://mlsanity.com/paper/1907.08948/) (2019)
- [Unsupervised Multi-modal Neural Machine Translation](https://mlsanity.com/paper/1811.11365/) (2018)
- [Modulating and attending the source image during encoding improves Multimodal Translation](https://mlsanity.com/paper/1712.03449/) (2017)

## Built on by (most cited first, 2 of 2)

- [Chitranuvad: Adapting Multi-Lingual LLMs for Multimodal Translation](https://mlsanity.com/paper/2502.20420/) (2025)
- [Impact of Visual Context on Noisy Multimodal NMT: An Empirical Study for English to Indian Languages](https://mlsanity.com/paper/2308.16075/) (2023)

## Abstract (arXiv)

Neural Machine Translation (NMT) is a predominant machine translation technology nowadays because of its end-to-end trainable flexibility. However, NMT still struggles to translate properly in low-resource settings specifically on distant language pairs. One way to overcome this is to use the information from other modalities if available. The idea is that despite differences in languages, both the source and target language speakers see the same thing and the visual representation of both the source and target is the same, which can positively assist the system. Multimodal information can help the NMT system to improve the translation by removing ambiguity on some phrases or words. We participate in the 8th Workshop on Asian Translation (WAT - 2021) for English-Hindi multimodal translation task and achieve 42.47 and 37.50 BLEU points for Evaluation and Challenge subset, respectively.