# Zero-shot Dependency Parsing with Pre-trained Multilingual Sentence Representations

arXiv: https://arxiv.org/abs/1910.05479 · HTML: https://mlsanity.com/paper/1910.05479/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Oct 2019.

## Built on by (most cited first, 17 of 17)

- [Towards a Common Understanding of Contributing Factors for Cross-Lingual Transfer in Multilingual Language Models: A Review](https://mlsanity.com/paper/2305.16768/) (2023)
- [Role of Language Relatedness in Multilingual Fine-tuning of Language Models: A Case Study in Indo-Aryan Languages](https://mlsanity.com/paper/2109.10534/) (2021)
- [Cultural Compass: Predicting Transfer Learning Success in Offensive Language Detection with Cultural Features](https://mlsanity.com/paper/2310.06458/) (2023)
- [Analysing The Impact Of Linguistic Features On Cross-Lingual Transfer](https://mlsanity.com/paper/2105.05975/) (2021)
- [Meta-Learning for Fast Cross-Lingual Adaptation in Dependency Parsing](https://mlsanity.com/paper/2104.04736/) (2021)
- [Towards a Better Understanding of Variations in Zero-Shot Neural Machine Translation Performance](https://mlsanity.com/paper/2310.10385/) (2023)
- [mCSQA: Multilingual Commonsense Reasoning Dataset with Unified Creation Strategy by Language Models and Humans](https://mlsanity.com/paper/2406.04215/) (2024)
- [Modality Matching Matters: Calibrating Language Distances for Cross-Lingual Transfer in URIEL+](https://mlsanity.com/paper/2510.19217/) (2025)
- [Data-Efficient Cross-Lingual Transfer with Language-Specific Subnetworks](https://mlsanity.com/paper/2211.00106/) (2022)
- [Analysing Cross-Lingual Transfer in Low-Resourced African Named Entity Recognition](https://mlsanity.com/paper/2309.05311/) (2023)
- [Better Benchmarking LLMs for Zero-Shot Dependency Parsing](https://mlsanity.com/paper/2502.20866/) (2025)
- [Transfer Learning for Multi-lingual Tasks -- a Survey](https://mlsanity.com/paper/2110.02052/) (2021)
- [AfriSUD: A Dependency Treebank Collection for Evaluating Models on African Languages](https://mlsanity.com/paper/2606.12708/) (2026)
- [Self-Attentive Constituency Parsing for UCCA-based Semantic Parsing](https://mlsanity.com/paper/2110.00621/) (2021)
- [Improving Low-Resource Cross-lingual Parsing with Expected Statistic Regularization](https://mlsanity.com/paper/2210.09428/) (2022)
- [CCAE: A Corpus of Chinese-based Asian Englishes](https://mlsanity.com/paper/2310.05381/) (2023)
- [Zero-shot Dependency Parsing with Unsupervised Cross-Lingual Bootstrapping](https://mlsanity.com/paper/2609.37883/) (2026)

## Abstract (arXiv)

We investigate whether off-the-shelf deep bidirectional sentence representations trained on a massively multilingual corpus (multilingual BERT) enable the development of an unsupervised universal dependency parser. This approach only leverages a mix of monolingual corpora in many languages and does not require any translation data making it applicable to low-resource languages. In our experiments we outperform the best CoNLL 2018 language-specific systems in all of the shared task's six truly low-resource languages while using a single system. However, we also find that (i) parsing accuracy still varies dramatically when changing the training languages and (ii) in some target languages zero-shot transfer fails under all tested conditions, raising concerns on the 'universality' of the whole approach.