# Zero-Shot Cross-Lingual Transfer with Meta Learning

arXiv: https://arxiv.org/abs/2003.02739 · HTML: https://mlsanity.com/paper/2003.02739/

## Key facts

- Cited by 66 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Mar 2020.

## Built on by (most cited first, 30 of 66)

- [CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLP](https://mlsanity.com/paper/2104.08835/) (2021)
- [Multilingual LLMs are Better Cross-lingual In-context Learners with Alignment](https://mlsanity.com/paper/2305.05940/) (2023)
- [LLM-powered Data Augmentation for Enhanced Cross-lingual Performance](https://mlsanity.com/paper/2305.14288/) (2023)
- [Frustratingly Easy Label Projection for Cross-lingual Transfer](https://mlsanity.com/paper/2211.15613/) (2022)
- [Zero-shot cross-lingual transfer language selection using linguistic similarity](https://mlsanity.com/paper/2301.13720/) (2023)
- [Meta Learning for Natural Language Processing: A Survey](https://mlsanity.com/paper/2205.01500/) (2022)
- [Why Does Zero-Shot Cross-Lingual Generation Fail? An Explanation and a Solution](https://mlsanity.com/paper/2305.17325/) (2023)
- [Improving Zero-Shot Cross-Lingual Transfer Learning via Robust Training](https://mlsanity.com/paper/2104.08645/) (2021)
- [The Emergence of Abstract Thought in Large Language Models Beyond Any Language](https://mlsanity.com/paper/2506.09890/) (2025)
- [Task Aligned Generative Meta-learning for Zero-shot Learning](https://mlsanity.com/paper/2103.02185/) (2021)
- [Zero-Shot Cross-lingual Semantic Parsing](https://mlsanity.com/paper/2104.07554/) (2021)
- [Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2104.08570/) (2021)
- [Should we Stop Training More Monolingual Models, and Simply Use Machine Translation Instead?](https://mlsanity.com/paper/2104.10441/) (2021)
- [Sequential Reptile: Inter-Task Gradient Alignment for Multilingual Learning](https://mlsanity.com/paper/2110.02600/) (2021)
- [Multilingual and cross-lingual document classification: A meta-learning approach](https://mlsanity.com/paper/2101.11302/) (2021)
- [Same Neurons, Different Languages: Probing Morphosyntax in Multilingual Pre-trained Models](https://mlsanity.com/paper/2205.02023/) (2022)
- [Language Models' Factuality Depends on the Language of Inquiry](https://mlsanity.com/paper/2502.17955/) (2025)
- [Meta-Learning for Effective Multi-task and Multilingual Modelling](https://mlsanity.com/paper/2101.10368/) (2021)
- [Examining Modularity in Multilingual LMs via Language-Specialized Subnetworks](https://mlsanity.com/paper/2311.08273/) (2023)
- [Meta-Learning a Cross-lingual Manifold for Semantic Parsing](https://mlsanity.com/paper/2209.12577/) (2022)
- [Model-Agnostic Meta-Learning for Multilingual Hate Speech Detection](https://mlsanity.com/paper/2303.02513/) (2023)
- [Meta-Learning for Fast Cross-Lingual Adaptation in Dependency Parsing](https://mlsanity.com/paper/2104.04736/) (2021)
- [X-METRA-ADA: Cross-lingual Meta-Transfer Learning Adaptation to Natural Language Understanding and Question Answering](https://mlsanity.com/paper/2104.09696/) (2021)
- [Does Typological Blinding Impede Cross-Lingual Sharing?](https://mlsanity.com/paper/2101.11888/) (2021)
- [CLUES: Few-Shot Learning Evaluation in Natural Language Understanding](https://mlsanity.com/paper/2111.02570/) (2021)
- [Fine-Tuning BERT with Character-Level Noise for Zero-Shot Transfer to Dialects and Closely-Related Languages](https://mlsanity.com/paper/2303.17683/) (2023)
- [CreoleVal: Multilingual Multitask Benchmarks for Creoles](https://mlsanity.com/paper/2310.19567/) (2023)
- [Attribute-Modulated Generative Meta Learning for Zero-Shot Classification](https://mlsanity.com/paper/2104.10857/) (2021)
- [Nearest Neighbour Few-Shot Learning for Cross-lingual Classification](https://mlsanity.com/paper/2109.02221/) (2021)
- [Wine is Not v i n. -- On the Compatibility of Tokenizations Across Languages](https://mlsanity.com/paper/2109.05772/) (2021)

## Abstract (arXiv)

Learning what to share between tasks has been a topic of great importance recently, as strategic sharing of knowledge has been shown to improve downstream task performance. This is particularly important for multilingual applications, as most languages in the world are under-resourced. Here, we consider the setting of training models on multiple different languages at the same time, when little or no data is available for languages other than English. We show that this challenging setup can be approached using meta-learning, where, in addition to training a source language model, another model learns to select which training instances are the most beneficial to the first. We experiment using standard supervised, zero-shot cross-lingual, as well as few-shot cross-lingual settings for different natural language understanding tasks (natural language inference, question answering). Our extensive experimental setup demonstrates the consistent effectiveness of meta-learning for a total of 15 languages. We improve upon the state-of-the-art for zero-shot and few-shot NLI (on MultiNLI and XNLI) and QA (on the MLQA dataset). A comprehensive error analysis indicates that the correlation of typological features between languages can partly explain when parameter sharing learned via meta-learning is beneficial.