Image-Text Relation Prediction for Multilingual Tweets
On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning
Abstract (arXiv)
Various social networks have been allowing media uploads for over a decade now. Still, it has not always been clear what is their relation with the posted text or even if there is any at all. In this work, we explore how multilingual vision-language models tackle the task of image-text relation prediction in different languages, and construct a dedicated balanced benchmark data set from Twitter posts in Latvian along with their manual translations into English. We compare our results to previous work and show that the more recently released vision-language model checkpoints are becoming increasingly capable at this task, but there is still much room for further improvement.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 6 of its 11 references that are in the map. First posted on arXiv 8 May 2025.
Builds on
6 of 11 references.
Closest ancestors · 2
Same direction, newest first
- 2024WorldCuisines: A Massive-Scale Benchmark for Multilingual and Multicultural Visual Question Answering on Global Cuisines16 Oct
- 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond24 Aug
Other references · 4
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2020COMET: A Neural Framework for MT Evaluationcited by 1,024
- 2024VL-ICL Bench: The Devil in the Details of Multimodal In-Context Learningcited by 34
- 2024Annotations for Exploring Food Tweets From Multiple Aspectscited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024MuLan: Adapting Multilingual Diffusion Models for Hundreds of Languages with Negligible Cost
- 2024Constructing Multilingual Visual-Text Datasets Revealing Visual Multilingual Ability of Vision Language Models
- 2025Boosting Text-To-Image Generation via Multilingual Prompting in Large Multimodal Models
- 2025Rethinking Multilingual Vision-Language Translation: Dataset, Evaluation, and Adaptation
- 2021Facebook AI WMT21 News Translation Task Submission
- 2025Exploring Vision Language Models for Multimodal and Multilingual Stance Detection
- 2025Multimodal Large Language Models for Text-rich Image Understanding: A Comprehensive Review
- 2024MMRA: A Benchmark for Evaluating Multi-Granularity and Multi-Image Relational Association Capabilities in Large Visual Language Models