# Low-Resource Parsing with Crosslingual Contextualized Representations

arXiv: https://arxiv.org/abs/1909.08744 · HTML: https://mlsanity.com/paper/1909.08744/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Sep 2019.

## Built on by (most cited first, 5 of 5)

- [An Empirical Survey of Data Augmentation for Limited Data Learning in NLP](https://mlsanity.com/paper/2106.07499/) (2021)
- [Evaluating GPT-4 and ChatGPT on Japanese Medical Licensing Examinations](https://mlsanity.com/paper/2303.18027/) (2023)
- [One Question Answering Model for Many Languages with Cross-lingual Dense Passage Retrieval](https://mlsanity.com/paper/2107.11976/) (2021)
- [Not All Linearizations Are Equally Data-Hungry in Sequence Labeling Parsing](https://mlsanity.com/paper/2108.07556/) (2021)
- [Cross-lingual Inflection as a Data Augmentation Method for Parsing](https://mlsanity.com/paper/2205.09350/) (2022)

## Abstract (arXiv)

Despite advances in dependency parsing, languages with small treebanks still present challenges. We assess recent approaches to multilingual contextual word representations (CWRs), and compare them for crosslingual transfer from a language with a large treebank to a language with a small or nonexistent treebank, by sharing parameters between languages in the parser itself. We experiment with a diverse selection of languages in both simulated and truly low-resource scenarios, and show that multilingual CWRs greatly facilitate low-resource dependency parsing even without crosslingual supervision such as dictionaries or parallel text. Furthermore, we examine the non-contextual part of the learned language models (which we call a "decontextual probe") to demonstrate that polyglot language models better encode crosslingual lexical correspondence compared to aligned monolingual language models. This analysis provides further evidence that polyglot training is an effective approach to crosslingual transfer.