mlsanityData to 7 Oct 2026

Meta-Learning for Effective Multi-task and Multilingual Modelling

arXiv 2101.10368 ↗First posted 25 Jan 2021cs.CLcited by 13

Not on the map. Why?

Abstract (arXiv)

Natural language processing (NLP) tasks (e.g. question-answering in English) benefit from knowledge of other tasks (e.g. named entity recognition in English) and knowledge of other languages (e.g. question-answering in Spanish). Such shared representations are typically learned in isolation, either across tasks or across languages. In this work, we propose a meta-learning approach to learn the interactions between both tasks and languages. We also investigate the role of different sampling strategies used during meta-learning. We present experiments on five different tasks and six different languages from the XTREME multilingual benchmark dataset. Our meta-learned model clearly improves in performance compared to competitive baseline models that also include multi-task baselines. We also present zero-shot evaluations on unseen target languages to demonstrate the utility of our proposed model.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 33 of its 45 references that are in the map. First posted on arXiv 25 Jan 2021.

Builds on

33 of 45 references.

References · 33

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  5. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  6. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  7. 2017Decoupled Weight Decay Regularizationcited by 1,722
  8. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,345
  9. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  10. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  11. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 913
  12. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 738
21 more
  1. 2019How multilingual is Multilingual BERT?cited by 642
  2. 2018End-to-End Multi-Task Learning with Attentioncited by 533
  3. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
  4. 2019Multi-Task Deep Neural Networks for Natural Language Understandingcited by 349
  5. 2019MLQA: Evaluating Cross-lingual Extractive Question Answeringcited by 339
  6. 2019Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERTcited by 301
  7. 2020Universal Dependencies v2: An Evergrowing Multilingual Treebank Collectioncited by 269
  8. 2019PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identificationcited by 262
  9. 2019Massively Multilingual Neural Machine Translation in the Wild: Findings and Challengescited by 253
  10. 2019Massively Multilingual Neural Machine Translationcited by 246
  11. 2018The Natural Language Decathlon: Multitask Learning as Question Answeringcited by 235
  12. 2018Meta-Learning for Low-Resource Neural Machine Translationcited by 105
  13. 2020Balancing Training for Multilingual Neural Machine Translationcited by 71
  14. 2016Deep Multi-task Representation Learning: A Tensor Factorisation Approachcited by 69
  15. 2020Zero-Shot Cross-Lingual Transfer with Meta Learningcited by 66
  16. 2016A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Taskscited by 62
  17. 2019Investigating Meta-Learning Algorithms for Low-Resource Natural Language Understanding Taskscited by 48
  18. 2019Optimizing Data Usage via Differentiable Rewardscited by 48
  19. 2019Cross-Lingual Training for Automatic Question Generationcited by 23
  20. 2018Meta Multi-Task Learning for Sequence Modelingcited by 18
  21. 2018Scheduled Multi-Task Learning: From Syntax to Translationcited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 6; 2023: 1; 2024: 1; 2025: 1; 2026: 1.

13 papers.

Sort:

By area:

  1. 2022Meta Learning for Natural Language Processing: A Surveycited by 26
  2. 2025Aligning LLMs for Multilingual Consistency in Enterprise Applicationscited by 10
  3. 2022Multi-Task Meta Learning: learn how to adapt to unseen taskscited by 8
  4. 2022EntityCS: Improving Zero-Shot Cross-lingual Transfer with Entity-Centric Code Switchingcited by 8
  5. 2021Sharing to learn and learning to share; Fitting together Meta-Learning, Multi-Task Learning, and Transfer Learning: A meta reviewcited by 6
  6. 2021The Effectiveness of Intermediate-Task Training for Code-Switched Natural Language Understandingcited by 4
  7. 2022Meta-XNLG: A Meta-Learning Approach Based on Language Clustering for Zero-Shot Cross-Lingual Transfer and Generationcited by 4
  8. 2022m4Adapter: Multilingual Multi-Domain Adaptation for Machine Translation with a Meta-Adaptercited by 2
  9. 2021WARM: A Weakly (+Semi) Supervised Model for Solving Math word Problemscited by 1
  10. 2023Multilingual Sentence-Level Semantic Search using Meta-Distillation Learningcited by 1
  11. 2022Persian Natural Language Inference: A Meta-learning approachcited by 0
  12. 2024Meta-RTL: Reinforcement-Based Meta-Transfer Learning for Low-Resource Commonsense Reasoningcited by 0
1 more
  1. 2026MAPLE: A Meta-learning Framework for Cross-Prompt Essay Scoringcited by 0