mlsanityData to 7 Oct 2026

X-METRA-ADA: Cross-lingual Meta-Transfer Learning Adaptation to Natural Language Understanding and Question Answering

arXiv 2104.09696 ↗First posted 20 Apr 2021cs.CLcited by 10

Not on the map. Why?

Abstract (arXiv)

Multilingual models, such as M-BERT and XLM-R, have gained increasing popularity, due to their zero-shot cross-lingual transfer learning capabilities. However, their generalization ability is still inconsistent for typologically diverse languages and across different benchmarks. Recently, meta-learning has garnered attention as a promising technique for enhancing transfer learning under low-resource scenarios: particularly for cross-lingual transfer in Natural Language Understanding (NLU). In this work, we propose X-METRA-ADA, a cross-lingual MEta-TRAnsfer learning ADAptation approach for NLU. Our approach adapts MAML, an optimization-based meta-learning approach, to learn to adapt to new languages. We extensively evaluate our framework on two challenging cross-lingual NLU tasks: multilingual task-oriented dialog and typologically diverse question answering. We show that our approach outperforms naive fine-tuning, reaching competitive performance on both tasks for most languages. Our analysis reveals that X-METRA-ADA can leverage limited data for faster adaptation.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 36 of its 44 references that are in the map. First posted on arXiv 20 Apr 2021.

Builds on

36 of 44 references.

References · 36

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  5. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  6. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  7. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 913
  8. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 738
  9. 2019How multilingual is Multilingual BERT?cited by 642
  10. 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 591
  11. 2019On the Cross-lingual Transferability of Monolingual Representationscited by 575
  12. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 486
24 more
  1. 2019MLQA: Evaluating Cross-lingual Extractive Question Answeringcited by 339
  2. 2019Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examplescited by 312
  3. 2019Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERTcited by 301
  4. 2020Universal Dependencies v2: An Evergrowing Multilingual Treebank Collectioncited by 269
  5. 2020XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generationcited by 221
  6. 2018Cross-Lingual Transfer Learning for Multilingual Task Oriented Dialogcited by 124
  7. 2018Meta-Learning for Low-Resource Neural Machine Translationcited by 105
  8. 2019Unicoder: A Universal Language Encoder by Pre-training with Multiple Cross-lingual Taskscited by 96
  9. 2019FewRel 2.0: Towards More Challenging Few-Shot Relation Classificationcited by 94
  10. 2020learn2learn: A Library for Meta-Learning Researchcited by 69
  11. 2020Zero-Shot Cross-Lingual Transfer with Meta Learningcited by 66
  12. 2019Investigating Meta-Learning Algorithms for Low-Resource Natural Language Understanding Taskscited by 48
  13. 2019Cross-Lingual BERT Transformation for Zero-Shot Dependency Parsingcited by 46
  14. 2018A Corpus for Multilingual Document Classification in Eight Languagescited by 42
  15. 2019Attention-Informed Mixed-Language Training for Zero-shot Cross-lingual Task-oriented Dialogue Systemscited by 42
  16. 2020XPersona: Evaluating Multilingual Personalized Chatbotcited by 36
  17. 2019Evaluating the Cross-Lingual Effectiveness of Massively Multilingual Neural Machine Translationcited by 27
  18. 2020Cross-lingual Information Retrieval with BERTcited by 27
  19. 2019Enhanced Meta-Learning for Cross-lingual Named Entity Recognition with Minimal Resourcescited by 25
  20. 2019Multi-lingual Intent Detection and Slot Filling in a Joint BERT-based Modelcited by 24
  21. 2020Meta-Transfer Learning for Code-Switched Speech Recognitioncited by 22
  22. 2019Zero-shot Cross-lingual Dialogue Systems with Transferable Latent Variablescited by 21
  23. 2019Zero-Resource Cross-Lingual Named Entity Recognitioncited by 18
  24. 2018Churn Intent Detection in Multilingual Chatbot Conversations and Social Mediacited by 2

Built on by

Cited by year

Citing papers per year: 2022: 5; 2023: 2; 2024: 3.

10 papers.

  1. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 97
  2. 2022Meta Learning for Natural Language Processing: A Surveycited by 26
  3. 2022Cross-lingual Lifelong Learningcited by 9
  4. 2022Meta-XNLG: A Meta-Learning Approach Based on Language Clustering for Zero-Shot Cross-Lingual Transfer and Generationcited by 4
  5. 2023CCT-Code: Cross-Consistency Training for Multilingual Clone Detection and Code Searchcited by 4
  6. 2024MIDAS: Multi-level Intent, Domain, And Slot Knowledge Distillation for Multi-turn NLUcited by 2
  7. 2022COOL, a Context Outlooker, and its Application to Question Answering and other Natural Language Processing Taskscited by 1
  8. 2023Multilingual Sentence-Level Semantic Search using Meta-Distillation Learningcited by 1
  9. 2024Adapting Mental Health Prediction Tasks for Cross-lingual Learning via Meta-Training and In-context Learning with Large Language Modelcited by 1
  10. 2024UniPSDA: Unsupervised Pseudo Semantic Data Augmentation for Zero-Shot Cross-Lingual Natural Language Understandingcited by 0