Meta-learning For Vision-and-language Cross-lingual Transfer
Not on the map. Why?
Abstract (arXiv)
Current pre-trained vison-language models (PVLMs) achieve excellent performance on a range of multi-modal datasets. Recent work has aimed at building multilingual models, and a range of novel multilingual multi-modal datasets have been proposed. Current PVLMs typically perform poorly on these datasets when used for multi-modal zero-shot or few-shot cross-lingual transfer, especially for low-resource languages. To alleviate this problem, we propose a novel meta-learning fine-tuning framework. Our framework makes current PVLMs rapidly adaptive to new languages in vision-language scenarios by designing MAML in a cross-lingual multi-modal manner. Experiments show that our method boosts the performance of current state-of-the-art PVLMs in both zero-shot and few-shot cross-lingual transfer on a range of vision-language understanding tasks and datasets (XVNLI, xGQA, MaRVL, xFlicker&Co)
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 29 of its 38 references that are in the map. First posted on arXiv 24 May 2023.
Builds on
29 of 38 references.
References · 29
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
- 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
- 2016Matching Networks for One Shot Learningcited by 2,246
- 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
- 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
- 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,352
- 2020Gradient Surgery for Multi-Task Learningcited by 1,210
- 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
17 more
- 2014Show and Tell: A Neural Image Caption Generatorcited by 877
- 2017Learning to Generalize: Meta-Learning for Domain Generalizationcited by 762
- 2020Contrastive Learning of Medical Visual Representations from Paired Images and Textcited by 636
- 2018A Corpus for Reasoning About Natural Language Grounded in Photographscited by 457
- 2018Meta-Learning for Semi-Supervised Few-Shot Classificationcited by 456
- 2016Multi30K: Multilingual English-German Image Descriptionscited by 250
- 2019Visual Entailment: A Novel Task for Fine-Grained Image Understandingcited by 250
- 2021Visually Grounded Reasoning across Languages and Culturescited by 188
- 2018Meta-Learning for Low-Resource Neural Machine Translationcited by 105
- 2021xGQA: Cross-Lingual Visual Question Answeringcited by 72
- 2020Zero-Shot Cross-Lingual Transfer with Meta Learningcited by 66
- 2021UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-trainingcited by 64
- 2022IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languagescited by 54
- 2020Meta-Learning for Domain Generalization in Semantic Parsingcited by 41
- 2017Image Pivoting for Learning Multilingual Multimodal Representationscited by 17
- 2022Multilingual Multimodal Learning with Machine Translated Textcited by 16
- 2017Visual Reasoning with Natural Languagecited by 2
Built on by
Citing papers per year: 2025: 3.
3 papers.