mlsanityData to 7 Oct 2026

Curriculum Script Distillation for Multilingual Visual Question Answering

arXiv 2301.07227 ↗First posted 17 Jan 2023cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Pre-trained models with dual and cross encoders have shown remarkable success in propelling the landscape of several tasks in vision and language in Visual Question Answering (VQA). However, since they are limited by the requirements of gold annotated data, most of these advancements do not see the light of day in other languages beyond English. We aim to address this problem by introducing a curriculum based on the source and target language translations to finetune the pre-trained models for the downstream task. Experimental results demonstrate that script plays a vital role in the performance of these models. Specifically, we show that target languages that share the same script perform better (~6%) than other languages and mixed-script code-switched languages perform better than their counterparts (~5-12%).

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 21 of its 28 references that are in the map. First posted on arXiv 17 Jan 2023.

Builds on

21 of 28 references.

References · 21

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  5. 2015VQA: Visual Question Answeringcited by 2,489
  6. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  7. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  8. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  9. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  10. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,291
  11. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
  12. 2019Cross-lingual Language Model Pretrainingcited by 1,088
9 more
  1. 2021MERLOT: Multimodal Neural Script Knowledge Modelscited by 311
  2. 2021CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representationcited by 229
  3. 2015Are You Talking to a Machine? Dataset and Methods for Multilingual Image Question Answeringcited by 80
  4. 2021Towards Developing a Multilingual and Code-Mixed Visual Question Answering System by Knowledge Distillationcited by 13
  5. 2022Delving Deeper into Cross-lingual Visual Question Answeringcited by 8
  6. 2021ViTA: Visual-Linguistic Translation by Aligning Object Tagscited by 5
  7. 2021Switch Point biased Self-Training: Re-purposing Pretrained Models for Code-Switchingcited by 5
  8. 2020Denoising Large-Scale Image Captioning from Alt-text Data using Content Selection Modelscited by 2
  9. 2022Multilingual Multimodality: A Taxonomical Survey of Datasets, Techniques, Challenges and Opportunitiescited by 1

Built on by

Not cited yet.