# Zero-Shot Dependency Parsing with Worst-Case Aware Automated Curriculum Learning

arXiv: https://arxiv.org/abs/2203.08555 · HTML: https://mlsanity.com/paper/2203.08555/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 37 references that are in the map.
- First posted on arXiv 16 Mar 2022.

## Builds on (most cited first, 16 of 16)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [How multilingual is Multilingual BERT?](https://mlsanity.com/paper/1906.01502/) (2019)
- [AllenNLP: A Deep Semantic Natural Language Processing Platform](https://mlsanity.com/paper/1803.07640/) (2018)
- [Deep Biaffine Attention for Neural Dependency Parsing](https://mlsanity.com/paper/1611.01734/) (2016)
- [Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT](https://mlsanity.com/paper/1904.09077/) (2019)
- [Universal Dependencies v2: An Evergrowing Multilingual Treebank Collection](https://mlsanity.com/paper/2004.10643/) (2020)
- [Automated Curriculum Learning for Neural Networks](https://mlsanity.com/paper/1704.03003/) (2017)
- [Fairness risk measures](https://mlsanity.com/paper/1901.08665/) (2019)
- [UDapter: Language Adaptation for Truly Universal Dependency Parsing](https://mlsanity.com/paper/2004.14327/) (2020)
- [Zero-Shot Cross-Lingual Transfer with Meta Learning](https://mlsanity.com/paper/2003.02739/) (2020)
- [Massive Choice, Ample Tasks (MaChAmp): A Toolkit for Multi-task Learning in NLP](https://mlsanity.com/paper/2005.14672/) (2020)
- [Minimax and Neyman-Pearson Meta-Learning for Outlier Languages](https://mlsanity.com/paper/2106.01051/) (2021)
- [Worst-Case-Aware Curriculum Learning for Zero and Few Shot Transfer](https://mlsanity.com/paper/2009.11138/) (2020)
- [Soft Layer Selection with Meta-Learning for Zero-Shot Cross-Lingual Transfer](https://mlsanity.com/paper/2107.09840/) (2021)

## Built on by (most cited first, 5 of 5)

- [Challenges and Strategies in Cross-Cultural NLP](https://mlsanity.com/paper/2203.10020/) (2022)
- [CLNode: Curriculum Learning for Node Classification](https://mlsanity.com/paper/2206.07258/) (2022)
- [The Zeno's Paradox of `Low-Resource' Languages](https://mlsanity.com/paper/2410.20817/) (2024)
- [Are Structural Concepts Universal in Transformer Language Models? Towards Interpretable Cross-Lingual Generalization](https://mlsanity.com/paper/2310.12794/) (2023)
- [Zero-shot Dependency Parsing with Unsupervised Cross-Lingual Bootstrapping](https://mlsanity.com/paper/2609.37883/) (2026)

## Abstract (arXiv)

Large multilingual pretrained language models such as mBERT and XLM-RoBERTa have been found to be surprisingly effective for cross-lingual transfer of syntactic parsing models (Wu and Dredze 2019), but only between related languages. However, source and training languages are rarely related, when parsing truly low-resource languages. To close this gap, we adopt a method from multi-task learning, which relies on automated curriculum learning, to dynamically optimize for parsing performance on outlier languages. We show that this approach is significantly better than uniform and size-proportional sampling in the zero-shot setting.