mlsanityData to 14 Oct 2026

Phylogeny-Inspired Adaptation of Multilingual Models to New Languages

arXiv 2205.09634 ↗First posted 19 May 2022cs.CLcited by 24

Not on the map. Why?

Abstract (arXiv)

Large pretrained multilingual models, trained on dozens of languages, have delivered promising results due to cross-lingual learning capabilities on variety of language tasks. Further adapting these models to specific languages, especially ones unseen during pre-training, is an important goal towards expanding the coverage of language technologies. In this study, we show how we can use language phylogenetic information to improve cross-lingual transfer leveraging closely related languages in a structured, linguistically-informed manner. We perform adapter-based training on languages from diverse language families (Germanic, Uralic, Tupian, Uto-Aztecan) and evaluate on both syntactic and semantic tasks, obtaining more than 20% relative performance improvements over strong commonly used baselines, especially on languages unseen during pre-training.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Builds on 21 of its 42 references that are in the map. First posted on arXiv 19 May 2022.

Builds on

21 of 42 references.

References · 21

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  3. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  5. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 915
  6. 2019How multilingual is Multilingual BERT?cited by 645
  7. 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 445
  8. 2020AdapterHub: A Framework for Adapting Transformerscited by 444
  9. 2016Deep Biaffine Attention for Neural Dependency Parsingcited by 334
  10. 2018Characterizing and Avoiding Negative Transfercited by 229
  11. 2022Lifting the Curse of Multilinguality by Pre-training Modular Transformerscited by 146
  12. 2020When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Modelscited by 131
9 more
  1. 2020UDapter: Language Adaptation for Truly Universal Dependency Parsingcited by 77
  2. 2021AmericasNLI: Evaluating Zero-shot Natural Language Understanding of Pretrained Multilingual Models in Truly Low-resource Languagescited by 50
  3. 2020Inducing Language-Agnostic Multilingual Representationscited by 48
  4. 2022Expanding Pretrained Models to Thousands More Languages via Lexicon-based Adaptationcited by 44
  5. 2020Is Supervised Syntactic Parsing Beneficial for Language Understanding? An Empirical Investigationcited by 34
  6. 2020Probing Multilingual BERT for Genetic and Typological Signalscited by 27
  7. 2022Match the Script, Adapt if Multilingual: Analyzing the Effect of Multilingual Pretraining on Cross-lingual Transferabilitycited by 26
  8. 2022Same Neurons, Different Languages: Probing Morphosyntax in Multilingual Pre-trained Modelscited by 14
  9. 2022Adaptable Adapterscited by 13

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 10; 2024: 7; 2025: 6.

24 papers.

Sort:

By area:

  1. 2022BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Promptingcited by 90
  2. 2023Modular Deep Learningcited by 83
  3. 2023SemEval-2023 Task 12: Sentiment Analysis for African Languages (AfriSenti-SemEval)cited by 41
  4. 2024Breaking the Curse of Multilinguality with Cross-lingual Expert Language Modelscited by 38
  5. 2024MaLA-500: Massive Language Adaptation of Large Language Modelscited by 28
  6. 2023Embedding structure matters: Comparing methods to adapt multilingual vocabularies to new languagescited by 16
  7. 2023Transfer to a Low-Resource Language via Close Relatives: The Case Study on Faroesecited by 15
  8. 2023Cross-Lingual Transfer with Target Language-Ready Task Adapterscited by 15
  9. 2023BIG-C: a Multimodal Multi-Purpose Dataset for Bembacited by 9
  10. 2024Targeted Multilingual Adaptation for Low-resource Language Familiescited by 9
  11. 2024An Efficient Approach for Studying Cross-Lingual Transfer in Multilingual Language Modelscited by 7
  12. 2023LIMIT: Language Identification, Misidentification, and Translation using Hierarchical Models in 350+ Languagescited by 6
12 more
  1. 2024The Impact of Language Adapters in Cross-Lingual Transfer for NLUcited by 6
  2. 2023To token or not to token: A Comparative Study of Text Representations for Cross-Lingual Transfercited by 4
  3. 2023Adapting to the Low-Resource Double-Bind: Investigating Low-Compute Methods on Low-Resource African Languagescited by 3
  4. 2024Data-Augmentation-Based Dialectal Adaptation for LLMscited by 3
  5. 2025Small Models, Big Impact: Efficient Corpus and Graph-Based Adaptation of Small Multilingual Language Models for Low-Resource Languagescited by 3
  6. 2025Modality Matching Matters: Calibrating Language Distances for Cross-Lingual Transfer in URIEL+cited by 3
  7. 2023GMNLP at SemEval-2023 Task 12: Sentiment Analysis with Phylogeny-Based Adapterscited by 2
  8. 2024Train More Parameters But Mind Their Placement: Insights into Language Adaptation with PEFTcited by 2
  9. 2025Sparse Subnetwork Enhancement for Underrepresented Languages in Large Language Modelscited by 1
  10. 2025How to Tune a Multilingual Encoder Model for Germanic Languages: A Study of PEFT, Full Fine-Tuning, and Language Adapterscited by 0
  11. 2025UoB-NLP at SemEval-2025 Task 11: Leveraging Adapters for Multilingual and Cross-Lingual Emotion Detectioncited by 0
  12. 2025Limited-Resource Adapters Are Regularizers, Not Linguistscited by 0