# Image captioning in different languages

arXiv: https://arxiv.org/abs/2407.09495 · HTML: https://mlsanity.com/paper/2407.09495/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 63 references that are in the map.
- First posted on arXiv 31 May 2024.

## Builds on (most cited first, 19 of 19)

- [Multi30K: Multilingual English-German Image Descriptions](https://mlsanity.com/paper/1605.00459/) (2016)
- [Crossmodal-3600: A Massively Multilingual Multimodal Evaluation Dataset](https://mlsanity.com/paper/2205.12522/) (2022)
- [COCO-CN for Cross-Lingual Image Tagging, Captioning and Retrieval](https://mlsanity.com/paper/1805.08661/) (2018)
- [Findings of the Second Shared Task on Multimodal Machine Translation and Multilingual Image Description](https://mlsanity.com/paper/1710.07177/) (2017)
- [AI Challenger : A Large-scale Dataset for Going Deeper in Image Understanding](https://mlsanity.com/paper/1711.06475/) (2017)
- [STAIR Captions: Constructing a Large-Scale Japanese Image Caption Dataset](https://mlsanity.com/paper/1705.00823/) (2017)
- [Fluency-Guided Cross-Lingual Image Captioning](https://mlsanity.com/paper/1708.04390/) (2017)
- [Bridge Correlational Neural Networks for Multilingual Multimodal Representation Learning](https://mlsanity.com/paper/1510.03519/) (2015)
- [Multimodal Pivots for Image Caption Translation](https://mlsanity.com/paper/1601.03916/) (2016)
- [Hindi Visual Genome: A Dataset for Multimodal English-to-Hindi Machine Translation](https://mlsanity.com/paper/1907.08948/) (2019)
- [UIT-ViIC: A Dataset for the First Evaluation on Vietnamese Image Captioning](https://mlsanity.com/paper/2002.00175/) (2020)
- [Cross-linguistic differences and similarities in image descriptions](https://mlsanity.com/paper/1707.01736/) (2017)
- [Chittron: An Automatic Bangla Image Captioning System](https://mlsanity.com/paper/1809.00339/) (2018)
- [#PraCegoVer: A Large Dataset for Image Captioning in Portuguese](https://mlsanity.com/paper/2103.11474/) (2021)
- [Violet: A Vision-Language Model for Arabic Image Captioning with Gemini Decoder](https://mlsanity.com/paper/2311.08844/) (2023)
- [Bornon: Bengali Image Captioning with Transformer-based Deep learning approach](https://mlsanity.com/paper/2109.05218/) (2021)
- [BAN-Cap: A Multi-Purpose English-Bangla Image Descriptions Dataset](https://mlsanity.com/paper/2205.14462/) (2022)
- [Efficient Urdu Caption Generation using Attention based LSTM](https://mlsanity.com/paper/2008.01663/) (2020)
- [UIT-OpenViIC: A Novel Benchmark for Evaluating Image Captioning in Vietnamese](https://mlsanity.com/paper/2305.04166/) (2023)

## Abstract (arXiv)

This short position paper provides a manually curated list of non-English image captioning datasets (as of May 2024). Through this list, we can observe the dearth of datasets in different languages: only 23 different languages are represented. With the addition of the Crossmodal-3600 dataset (Thapliyal et al., 2022, 36 languages) this number increases somewhat, but still this number is small compared to the +/-500 institutional languages that are out there. This paper closes with some open questions for the field of Vision & Language.