mlsanityData to 7 Oct 2026

xGQA: Cross-Lingual Visual Question Answering

arXiv 2109.06082 ↗First posted 13 Sep 2021cs.CLcited by 72

Not on the map. Why?

Cited by 30 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent advances in multimodal vision and language modeling have predominantly focused on the English language, mostly due to the lack of multilingual multimodal datasets to steer modeling efforts. In this work, we address this gap and provide xGQA, a new multilingual evaluation benchmark for the visual question answering task. We extend the established English GQA dataset to 7 typologically diverse languages, enabling us to detect and explore crucial challenges in cross-lingual visual question answering. We further propose new adapter-based approaches to adapt multimodal transformer-based models to become multilingual, and -- vice versa -- multilingual models to become multimodal. Our proposed methods outperform current state-of-the-art multilingual multimodal models (e.g., M3P) in zero-shot cross-lingual settings, but the accuracy remains low across the board; a performance drop of around 38 accuracy points in target languages showcases the difficulty of zero-shot cross-lingual transfer for this task. Our results suggest that simple cross-lingual transfer of multimodal models yields latent multilingual multimodal misalignment, calling for more sophisticated methods for vision and multilingual language modeling.

Key facts. Cited by 72 papers in the map (data through 7 Oct 2026). Builds on 54 of its 86 references that are in the map. First posted on arXiv 13 Sep 2021.

Builds on

54 of 86 references.

References · 54

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  7. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  8. 2020End-to-End Object Detection with Transformerscited by 7,384
  9. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  10. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  11. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  12. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
38 more
  1. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  2. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  3. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,352
  4. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
  5. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
  6. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 811
  7. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 780
  8. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 738
  9. 2019How multilingual is Multilingual BERT?cited by 642
  10. 2019On the Cross-lingual Transferability of Monolingual Representationscited by 575
  11. 2017Learning multiple visual domains with residual adapterscited by 553
  12. 2018A Corpus for Reasoning About Natural Language Grounded in Photographscited by 457
  13. 2019Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-trainingcited by 451
  14. 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 443
  15. 2020AdapterHub: A Framework for Adapting Transformerscited by 441
  16. 2020Large-Scale Adversarial Training for Vision-and-Language Representation Learningcited by 327
  17. 2020K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapterscited by 318
  18. 2020How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Modelscited by 313
  19. 2019Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERTcited by 301
  20. 2021WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learningcited by 285
  21. 2016Multi30K: Multilingual English-German Image Descriptionscited by 250
  22. 2019Simple, Scalable Adaptation for Neural Machine Translationcited by 239
  23. 2020AdapterDrop: On the Efficiency of Adapters in Transformerscited by 223
  24. 2019Cross-Lingual Ability of Multilingual BERT: An Empirical Studycited by 182
  25. 2020UNKs Everywhere: Adapting Multilingual Language Models to New Scriptscited by 113
  26. 2021VinVL: Revisiting Visual Representations in Vision-Language Modelscited by 109
  27. 2021Thinking Fast and Slow: Efficient Text-to-Visual Retrieval with Transformerscited by 93
  28. 2015Are You Talking to a Machine? Dataset and Methods for Multilingual Image Question Answeringcited by 80
  29. 2020UDapter: Language Adaptation for Truly Universal Dependency Parsingcited by 77
  30. 2020Multimodal Pretraining Unmasked: A Meta-Analysis and a Unified Framework of Vision-and-Language BERTscited by 71
  31. 2021Vision-and-Language or Vision-for-Language? On Cross-Modal Influence in Multimodal Transformerscited by 71
  32. 2021UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-trainingcited by 64
  33. 2021What to Pre-Train on? Efficient Intermediate Task Selectioncited by 62
  34. 2020Common Sense or World Knowledge? Investigating Adapter-Based Knowledge Injection into Pretrained Transformerscited by 57
  35. 2021Lightweight Adapter Tuning for Multilingual Speech Translationcited by 53
  36. 2021Structural Adapters in Pretrained Language Models for AMR-to-text Generationcited by 48
  37. 2021Multilingual Multimodal Pre-training for Zero-Shot Cross-Lingual Transfer of Vision-Language Modelscited by 41
  38. 2021Retrieve Fast, Rerank Smart: Cooperative and Joint Approaches for Improved Cross-Modal Retrievalcited by 27

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 14; 2023: 15; 2024: 15; 2025: 19; 2026: 9.

72 papers.

Sort:

By area:

  1. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 762
  2. 2024CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmarkcited by 118
  3. 2024MTVQA: Benchmarking Multilingual Text-Centric Visual Question Answeringcited by 93
  4. 2023Modular Deep Learningcited by 82
  5. 2022One Country, 700+ Languages: NLP Challenges for Underrepresented Languages and Dialects in Indonesiacited by 65
  6. 2025Meta CLIP 2: A Worldwide Scaling Recipecited by 59
  7. 2024All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languagescited by 55
  8. 2022IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languagescited by 54
  9. 2023Large Multilingual Models Pivot Zero-Shot Multimodal Learning across Languagescited by 52
  10. 2023mBLIP: Efficient Bootstrapping of Multilingual Vision-LLMscited by 41
  11. 2024Gradient Routing: Masking Gradients to Localize Computation in Neural Networkscited by 28
  12. 2022Tackling Ambiguity with Images: Improved Multimodal Machine Translation and Contrastive Evaluationcited by 26
38 more
  1. 2024FoodieQA: A Multimodal Dataset for Fine-Grained Understanding of Chinese Food Culturecited by 25
  2. 2022Cross-View Language Modeling: Towards Unified Cross-Lingual Cross-Modal Pre-trainingcited by 24
  3. 2023From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunitiescited by 24
  4. 2023Robust Visual Question Answering: Datasets, Methods, and Future Challengescited by 23
  5. 2024JMMMU: A Japanese Massive Multi-discipline Multimodal Understanding Benchmark for Culture-aware Evaluationcited by 19
  6. 2022Multilingual Multimodal Learning with Machine Translated Textcited by 16
  7. 2024No Filter: Cultural and Socioeconomic Diversity in Contrastive Vision-Language Modelscited by 16
  8. 2024Multilingual Diversity Improves Vision-Language Representationscited by 14
  9. 2023HaVQA: A Dataset for Visual Question Answering and Multimodal Research in Hausa Languagecited by 12
  10. 2023Babel-ImageNet: Massively Multilingual Evaluation of Vision-and-Language Representationscited by 11
  11. 2025Evaluating Multimodal Language Models as Visual Assistants for Visually Impaired Userscited by 11
  12. 2024Conditional computation in neural networks: principles and research trendscited by 10
  13. 2025The Multilingual Divide and Its Impact on Global AI Safetycited by 10
  14. 2022MaXM: Towards Multilingual Visual Question Answeringcited by 9
  15. 2022WildQA: In-the-Wild Video Question Answeringcited by 9
  16. 2022Delving Deeper into Cross-lingual Visual Question Answeringcited by 8
  17. 2022Weakly Supervised Grounding for VQA in Vision-Language Transformerscited by 8
  18. 2023Semantic and Expressive Variation in Image Captions Across Languagescited by 8
  19. 2024mOSCAR: A Large-scale Multilingual and Multimodal Document-level Corpuscited by 8
  20. 2022ERNIE-UniX2: A Unified Cross-lingual Cross-modal Framework for Understanding and Generationcited by 7
  21. 2024Towards Cross-Lingual Explanation of Artwork in Large-scale Vision Language Modelscited by 7
  22. 2025IndicVisionBench: Benchmarking Cultural and Multilingual Understanding in VLMscited by 7
  23. 2023EVJVQA Challenge: Multilingual Visual Question Answeringcited by 6
  24. 2023Efficient Domain Adaptation of Sentence Embeddings Using Adapterscited by 6
  25. 2025Pearl: A Multimodal Culturally-Aware Arabic Instruction Datasetcited by 6
  26. 2022Improving the Cross-Lingual Generalisation in Visual Question Answeringcited by 5
  27. 2024Image Translation as Diffusion Visual Programmerscited by 5
  28. 2024ChitroJera: A Regionally Relevant Visual Question Answering Dataset for Banglacited by 5
  29. 2025MVL-SIB: A Massively Multilingual Vision-Language Benchmark for Cross-Modal Topical Matchingcited by 5
  30. 2025A Culturally-diverse Multilingual Multimodal Video Benchmark & Modelcited by 5
  31. 2025OASIS: A Multilingual and Multimodal Dataset for Culturally Grounded Spoken Visual QAcited by 5
  32. 2025Benchmarking and Boosting Multilingual Capabilities of LVLMs via OCR-Centric Reinforcement Learningcited by 4
  33. 2025CUS-QA: Local-Knowledge-Oriented Open-Ended Question Answering Datasetcited by 4
  34. 2022Generalizing Multimodal Pre-training into Multilingual via Language Acquisitioncited by 3
  35. 2023Meta-learning For Vision-and-language Cross-lingual Transfercited by 3
  36. 2024What is "Typological Diversity" in NLP?cited by 3
  37. 2024VLR-Bench: Multilingual Benchmark Dataset for Vision-Language Retrieval Augmented Generationcited by 3
  38. 2025POLYCHARTQA: Benchmarking Large Vision-Language Models with Multilingual Chart Question Answeringcited by 3

50 most cited of 72 All 72 (JSON)