mlsanityData to 14 Oct 2026

MaXM: Towards Multilingual Visual Question Answering

arXiv 2209.05401 ↗First posted 12 Sep 2022cs.CL cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

Visual Question Answering (VQA) has been primarily studied through the lens of the English language. Yet, tackling VQA in other languages in the same manner would require a considerable amount of resources. In this paper, we propose scalable solutions to multilingual visual question answering (mVQA), on both data and modeling fronts. We first propose a translation-based framework to mVQA data generation that requires much less human annotation efforts than the conventional approach of directly collection questions and answers. Then, we apply our framework to the multilingual captions in the Crossmodal-3600 dataset and develop an efficient annotation protocol to create MaXM, a test-only VQA benchmark in 7 diverse languages. Finally, we develop a simple, lightweight, and effective approach as well as benchmark state-of-the-art English and multilingual VQA models. We hope that our benchmark encourages further research on mVQA.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 59 of its 80 references that are in the map. First posted on arXiv 12 Sep 2022.

Builds on

59 of 80 references.

References · 59

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  5. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  7. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  8. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  9. 2015VQA: Visual Question Answeringcited by 2,506
  10. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,046
  11. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  12. 2019Towards VQA Models That Can Readcited by 1,718
38 more
  1. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  2. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  3. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  4. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,462
  5. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
  6. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
  7. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,121
  8. 2014Deep Visual-Semantic Alignments for Generating Image Descriptionscited by 968
  9. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 905
  10. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 858
  11. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 745
  12. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 710
  13. 2021SimVLM: Simple Visual Language Model Pretraining with Weak Supervisioncited by 647
  14. 2018From Recognition to Cognition: Visual Commonsense Reasoningcited by 556
  15. 2022GIT: A Generative Image-to-text Transformer for Vision and Languagecited by 537
  16. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 469
  17. 2018nocaps: novel object captioning at scalecited by 444
  18. 2019VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Researchcited by 414
  19. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 414
  20. 2021Unifying Vision-and-Language Tasks via Text Generationcited by 412
  21. 2020Asking and Answering Questions to Evaluate the Factual Consistency of Summariescited by 378
  22. 2017Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answeringcited by 339
  23. 2019Scene Text Visual Question Answeringcited by 331
  24. 2015Visual7W: Grounded Question Answering in Imagescited by 318
  25. 2021WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learningcited by 287
  26. 2016Multi30K: Multilingual English-German Image Descriptionscited by 250
  27. 2020Just Ask: Learning to Answer Questions from Millions of Narrated Videoscited by 248
  28. 2021Quality at a Glance: An Audit of Web-Crawled Multilingual Datasetscited by 243
  29. 2019Connecting Vision and Language with Localized Narrativescited by 226
  30. 2021Visually Grounded Reasoning across Languages and Culturescited by 189
  31. 2021RedCaps: web-curated image-text data created by the people, for the peoplecited by 180
  32. 2019Synthetic QA Corpora Generation with Roundtrip Consistencycited by 143
  33. 2015Exploring Models and Data for Image Question Answeringcited by 139
  34. 2021Q2: Evaluating Factual Consistency in Knowledge-Grounded Dialogues via Question Generation and Question Answeringcited by 132
  35. 2022Crossmodal-3600: A Massively Multilingual Multimodal Evaluation Datasetcited by 100
  36. 2017An Analysis of Visual Question Answering Algorithmscited by 90
  37. 2018COCO-CN for Cross-Lingual Image Tagging, Captioning and Retrievalcited by 80
  38. 2017Findings of the Second Shared Task on Multimodal Machine Translation and Multilingual Image Descriptioncited by 76

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 5; 2024: 2; 2025: 1.

9 papers.

  1. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 766
  2. 2024Pangea: A Fully Open Multilingual Multimodal LLM for 39 Languagescited by 64
  3. 2023From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunitiescited by 25
  4. 2023Robust Visual Question Answering: Datasets, Methods, and Future Challengescited by 23
  5. 2023OpenViVQA: Task, Dataset, and Multimodal Fusion Models for Visual Question Answering in Vietnamesecited by 20
  6. 2024No Filter: Cultural and Socioeconomic Diversity in Contrastive Vision-Language Modelscited by 16
  7. 2023Semantic and Expressive Variation in Image Captions Across Languagescited by 8
  8. 2023EVJVQA Challenge: Multilingual Visual Question Answeringcited by 6
  9. 2025Centurio: On Drivers of Multilingual Ability of Large Vision-Language Modelcited by 3