# UDapter: Language Adaptation for Truly Universal Dependency Parsing

arXiv: https://arxiv.org/abs/2004.14327 · HTML: https://mlsanity.com/paper/2004.14327/

## Key facts

- Cited by 77 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Apr 2020.

## Built on by (most cited first, 30 of 77)

- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [Compacter: Efficient Low-Rank Hypercomplex Adapter Layers](https://mlsanity.com/paper/2106.04647/) (2021)
- [Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworks](https://mlsanity.com/paper/2106.04489/) (2021)
- [Lifting the Curse of Multilinguality by Pre-training Modular Transformers](https://mlsanity.com/paper/2205.06266/) (2022)
- [Composable Sparse Fine-Tuning for Cross-Lingual Transfer](https://mlsanity.com/paper/2110.07560/) (2021)
- [Sustainable Modular Debiasing of Language Models](https://mlsanity.com/paper/2109.03646/) (2021)
- [Modular Deep Learning](https://mlsanity.com/paper/2302.11529/) (2023)
- [xGQA: Cross-Lingual Visual Question Answering](https://mlsanity.com/paper/2109.06082/) (2021)
- [Trankit: A Light-Weight Transformer-based Toolkit for Multilingual Natural Language Processing](https://mlsanity.com/paper/2101.03289/) (2021)
- [What to Pre-Train on? Efficient Intermediate Task Selection](https://mlsanity.com/paper/2104.08247/) (2021)
- [A Primer on Pretrained Multilingual Language Models](https://mlsanity.com/paper/2107.00676/) (2021)
- [PERFECT: Prompt-free and Efficient Few-shot Learning with Language Models](https://mlsanity.com/paper/2204.01172/) (2022)
- [Efficient Hierarchical Domain Adaptation for Pretrained Language Models](https://mlsanity.com/paper/2112.08786/) (2021)
- [Learning to Generate Task-Specific Adapters from Task Description](https://mlsanity.com/paper/2101.00420/) (2021)
- [Language and Task Arithmetic with Parameter-Efficient Layers for Zero-Shot Summarization](https://mlsanity.com/paper/2311.09344/) (2023)
- [MaLA-500: Massive Language Adaptation of Large Language Models](https://mlsanity.com/paper/2401.13303/) (2024)
- [Efficient Test Time Adapter Ensembling for Low-resource Language Varieties](https://mlsanity.com/paper/2109.04877/) (2021)
- [HyperMoE: Towards Better Mixture of Experts via Transferring Among Experts](https://mlsanity.com/paper/2402.12656/) (2024)
- [Culturally Aware and Adapted NLP: A Taxonomy and a Survey of the State of the Art](https://mlsanity.com/paper/2406.03930/) (2024)
- [Overlap-based Vocabulary Generation Improves Cross-lingual Transfer Among Related Languages](https://mlsanity.com/paper/2203.01976/) (2022)
- [Phylogeny-Inspired Adaptation of Multilingual Models to New Languages](https://mlsanity.com/paper/2205.09634/) (2022)
- [mmT5: Modular Multilingual Pre-Training Solves Source Language Hallucinations](https://mlsanity.com/paper/2305.14224/) (2023)
- [Multilingual Unsupervised Neural Machine Translation with Denoising Adapters](https://mlsanity.com/paper/2110.10472/) (2021)
- [Hyperdecoders: Instance-specific decoders for multi-task NLP](https://mlsanity.com/paper/2203.08304/) (2022)
- [Hyper-X: A Unified Hypernetwork for Multi-Task Multilingual Transfer](https://mlsanity.com/paper/2205.12148/) (2022)
- [Unveiling Linguistic Regions in Large Language Models](https://mlsanity.com/paper/2402.14700/) (2024)
- [URIEL+: Enhancing Linguistic Inclusion and Usability in a Typological and Multilingual Knowledge Base](https://mlsanity.com/paper/2409.18472/) (2024)
- [Exploiting Language Relatedness for Low Web-Resource Language Model Adaptation: An Indic Languages Study](https://mlsanity.com/paper/2106.03958/) (2021)
- [Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2104.08570/) (2021)
- [Example-based Hypernetworks for Out-of-Distribution Generalization](https://mlsanity.com/paper/2203.14276/) (2022)

## Abstract (arXiv)

Recent advances in multilingual dependency parsing have brought the idea of a truly universal parser closer to reality. However, cross-language interference and restrained model capacity remain major obstacles. To address this, we propose a novel multilingual task adaptation approach based on contextual parameter generation and adapter modules. This approach enables to learn adapters via language embeddings while sharing model parameters across languages. It also allows for an easy but effective integration of existing linguistic typology features into the parsing network. The resulting parser, UDapter, outperforms strong monolingual and multilingual baselines on the majority of both high-resource and low-resource (zero-shot) languages, showing the success of the proposed adaptation approach. Our in-depth analyses show that soft parameter sharing via typological features is key to this success.