# Meta-Learning for Fast Cross-Lingual Adaptation in Dependency Parsing

arXiv: https://arxiv.org/abs/2104.04736 · HTML: https://mlsanity.com/paper/2104.04736/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 42 references that are in the map.
- First posted on arXiv 10 Apr 2021.

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Meta-Learning in Neural Networks: A Survey](https://mlsanity.com/paper/2004.05439/) (2020)
- [Deep Biaffine Attention for Neural Dependency Parsing](https://mlsanity.com/paper/1611.01734/) (2016)
- [Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT](https://mlsanity.com/paper/1904.09077/) (2019)
- [75 Languages, 1 Model: Parsing Universal Dependencies Universally](https://mlsanity.com/paper/1904.02099/) (2019)
- [Learning and Evaluating General Linguistic Intelligence](https://mlsanity.com/paper/1901.11373/) (2019)
- [UDapter: Language Adaptation for Truly Universal Dependency Parsing](https://mlsanity.com/paper/2004.14327/) (2020)
- [Cross-Lingual Alignment of Contextual Word Embeddings, with Applications to Zero-shot Dependency Parsing](https://mlsanity.com/paper/1902.09492/) (2019)
- [Zero-Shot Cross-Lingual Transfer with Meta Learning](https://mlsanity.com/paper/2003.02739/) (2020)
- [From Zero to Hero: On the Limitations of Zero-Shot Cross-Lingual Transfer with Multilingual Transformers](https://mlsanity.com/paper/2005.00633/) (2020)
- [Natural Language to Structured Query Generation via Meta-Learning](https://mlsanity.com/paper/1803.02400/) (2018)
- [Many Languages, One Parser](https://mlsanity.com/paper/1602.01595/) (2016)
- [Enhanced Meta-Learning for Cross-lingual Named Entity Recognition with Minimal Resources](https://mlsanity.com/paper/1911.06161/) (2019)
- [Towards Better UD Parsing: Deep Contextualized Word Embeddings, Ensemble, and Treebank Concatenation](https://mlsanity.com/paper/1807.03121/) (2018)
- [Zero-shot Dependency Parsing with Pre-trained Multilingual Sentence Representations](https://mlsanity.com/paper/1910.05479/) (2019)
- [Learning to Learn to Disambiguate: Meta-Learning for Few-Shot Word Sense Disambiguation](https://mlsanity.com/paper/2004.14355/) (2020)
- [Learning to Learn Morphological Inflection for Resource-Poor Languages](https://mlsanity.com/paper/2004.13304/) (2020)

## Built on by (most cited first, 10 of 10)

- [Meta Learning for Natural Language Processing: A Survey](https://mlsanity.com/paper/2205.01500/) (2022)
- [Enhancing Healthcare through Large Language Models: A Study on Medical Question Answering](https://mlsanity.com/paper/2408.04138/) (2024)
- [The Zeno's Paradox of `Low-Resource' Languages](https://mlsanity.com/paper/2410.20817/) (2024)
- [Are Structural Concepts Universal in Transformer Language Models? Towards Interpretable Cross-Lingual Generalization](https://mlsanity.com/paper/2310.12794/) (2023)
- [Data-Efficient Cross-Lingual Transfer with Language-Specific Subnetworks](https://mlsanity.com/paper/2211.00106/) (2022)
- [Should Cross-Lingual AMR Parsing go Meta? An Empirical Assessment of Meta-Learning and Joint Learning AMR Parsing](https://mlsanity.com/paper/2410.03357/) (2024)
- [Multilingual Sentence-Level Semantic Search using Meta-Distillation Learning](https://mlsanity.com/paper/2309.08185/) (2023)
- [Incorporating Lexical and Syntactic Knowledge for Unsupervised Cross-Lingual Transfer](https://mlsanity.com/paper/2404.16627/) (2024)
- [Learning New Tasks from a Few Examples with Soft-Label Prototypes](https://mlsanity.com/paper/2210.17437/) (2022)
- [Zero-shot Dependency Parsing with Unsupervised Cross-Lingual Bootstrapping](https://mlsanity.com/paper/2609.37883/) (2026)

## Abstract (arXiv)

Meta-learning, or learning to learn, is a technique that can help to overcome resource scarcity in cross-lingual NLP problems, by enabling fast adaptation to new tasks. We apply model-agnostic meta-learning (MAML) to the task of cross-lingual dependency parsing. We train our model on a diverse set of languages to learn a parameter initialization that can adapt quickly to new languages. We find that meta-learning with pre-training can significantly improve upon the performance of language transfer and standard supervised learning baselines for a variety of unseen, typologically diverse, and low-resource languages, in a few-shot learning setup.