# Hindi Visual Genome: A Dataset for Multimodal English-to-Hindi Machine Translation

arXiv: https://arxiv.org/abs/1907.08948 · HTML: https://mlsanity.com/paper/1907.08948/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jul 2019.

## Built on by (most cited first, 16 of 16)

- [Universal Multimodal Representation for Language Understanding](https://mlsanity.com/paper/2301.03344/) (2023)
- [Hausa Visual Genome: A Dataset for Multi-Modal English to Hausa Machine Translation](https://mlsanity.com/paper/2205.01133/) (2022)
- [BIG-C: a Multimodal Multi-Purpose Dataset for Bemba](https://mlsanity.com/paper/2305.17202/) (2023)
- [Chitranuvad: Adapting Multi-Lingual LLMs for Multimodal Translation](https://mlsanity.com/paper/2502.20420/) (2025)
- [ViTA: Visual-Linguistic Translation by Aligning Object Tags](https://mlsanity.com/paper/2106.00250/) (2021)
- [A Multimodal In-Context Tuning Approach for E-Commerce Product Description Generation](https://mlsanity.com/paper/2402.13587/) (2024)
- [Large Multimodal Models for Low-Resource Languages: A Survey](https://mlsanity.com/paper/2502.05568/) (2025)
- [Parallel Corpora for Machine Translation in Low-resource Indic Languages: A Comprehensive Review](https://mlsanity.com/paper/2503.04797/) (2025)
- [IITP at WAT 2021: System description for English-Hindi Multimodal Translation Task](https://mlsanity.com/paper/2107.01656/) (2021)
- [EMMT: A simultaneous eye-tracking, 4-electrode EEG and audio corpus for multi-modal reading and translation scenarios](https://mlsanity.com/paper/2204.02905/) (2022)
- [Silo NLP's Participation at WAT2022](https://mlsanity.com/paper/2208.01296/) (2022)
- [Impact of Visual Context on Noisy Multimodal NMT: An Empirical Study for English to Indian Languages](https://mlsanity.com/paper/2308.16075/) (2023)
- [Brotherhood at WMT 2024: Leveraging LLM-Generated Contextual Conversations for Cross-Lingual Image Captioning](https://mlsanity.com/paper/2409.15052/) (2024)
- [Comprehension of Multilingual Expressions Referring to Target Objects in Visual Inputs](https://mlsanity.com/paper/2511.11427/) (2025)
- [Image captioning in different languages](https://mlsanity.com/paper/2407.09495/) (2024)
- [A Picture is Worth a Thousand (Correct) Captions: A Vision-Guided Judge-Corrector System for Multimodal Machine Translation](https://mlsanity.com/paper/2511.07010/) (2025)

## Abstract (arXiv)

Visual Genome is a dataset connecting structured image information with English language. We present ``Hindi Visual Genome'', a multimodal dataset consisting of text and images suitable for English-Hindi multimodal machine translation task and multimodal research. We have selected short English segments (captions) from Visual Genome along with associated images and automatically translated them to Hindi with manual post-editing which took the associated images into account. We prepared a set of 31525 segments, accompanied by a challenge test set of 1400 segments. This challenge test set was created by searching for (particularly) ambiguous English words based on the embedding similarity and manually selecting those where the image helps to resolve the ambiguity. Our dataset is the first for multimodal English-Hindi machine translation, freely available for non-commercial research purposes. Our Hindi version of Visual Genome also allows to create Hindi image labelers or other practical tools. Hindi Visual Genome also serves in Workshop on Asian Translation (WAT) 2019 Multi-Modal Translation Task.