mlsanityData to 7 Oct 2026

Meta Learning for Natural Language Processing: A Survey

arXiv 2205.01500 ↗First posted 3 May 2022cs.CL cs.AI cs.LGcited by 26

Not on the map. Why?

Abstract (arXiv)

Deep learning has been the mainstream technique in natural language processing (NLP) area. However, the techniques require many labeled data and are less generalizable across domains. Meta-learning is an arising field in machine learning studying approaches to learn better learning algorithms. Approaches aim at improving algorithms in various aspects, including data efficiency and generalizability. Efficacy of approaches has been shown in many NLP tasks, but there is no systematic survey of these approaches in NLP, which hinders more researchers from joining the field. Our goal with this survey paper is to offer researchers pointers to relevant meta-learning works in NLP and attract more attention from the NLP community to drive future innovation. This paper first introduces the general concepts of meta-learning and the common approaches. Then we summarize task construction settings and application of meta-learning for various NLP problems and review the development of meta-learning in NLP community.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 125 of its 164 references that are in the map. First posted on arXiv 3 May 2022.

Builds on

125 of 164 references.

References · 125

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  5. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  6. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  7. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  8. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  9. 2016Learning without Forgettingcited by 2,465
  10. 2016Matching Networks for One Shot Learningcited by 2,246
  11. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  12. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
38 more
  1. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  2. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,139
  3. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,097
  4. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,094
  5. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  6. 2020Meta-Learning in Neural Networks: A Surveycited by 944
  7. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
  8. 2017Learning to Generalize: Meta-Learning for Domain Generalizationcited by 762
  9. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 738
  10. 2016Learning to learn by gradient descent by gradient descentcited by 616
  11. 2021Editing Factual Knowledge in Language Modelscited by 548
  12. 2018Progress & Compress: A scalable framework for continual learningcited by 516
  13. 2020MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devicescited by 476
  14. 2021Gradient Projection Memory for Continual Learningcited by 338
  15. 2019Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examplescited by 312
  16. 2018How to train your MAMLcited by 306
  17. 2018Meta-Learning: A Surveycited by 254
  18. 2019Episodic Memory in Lifelong Language Learningcited by 188
  19. 2021SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processingcited by 177
  20. 2021Meta-learning via Language Model In-context Tuningcited by 163
  21. 2019LAMOL: LAnguage MOdeling for Lifelong Language Learningcited by 156
  22. 2019A review of domain adaptation without target labelscited by 154
  23. 2020Editable Neural Networkscited by 154
  24. 2021CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLPcited by 142
  25. 2020HAT: Hardware-Aware Transformers for Efficient Natural Language Processingcited by 136
  26. 2019Meta Label Correction for Noisy Label Learningcited by 108
  27. 2018Meta-Learning for Low-Resource Neural Machine Translationcited by 105
  28. 2020A Survey of Deep Meta-Learningcited by 99
  29. 2019FewRel 2.0: Towards More Challenging Few-Shot Relation Classificationcited by 94
  30. 2019Personalizing Dialogue Agents via Meta-Learningcited by 83
  31. 2020Few-shot Slot Tagging with Collapsed Dependency Transfer and Label-enhanced Task-adaptive Projection Networkcited by 83
  32. 2018One-Shot Relational Learning for Knowledge Graphscited by 78
  33. 2019Meta Relational Learning for Few-Shot Link Prediction in Knowledge Graphscited by 73
  34. 2020Low-Resource Domain Adaptation for Compositional Task-Oriented Semantic Parsingcited by 72
  35. 2020Balancing Training for Multilingual Neural Machine Translationcited by 71
  36. 2021FLEX: Unifying Evaluation for Few-Shot NLPcited by 67
  37. 2020Zero-Shot Cross-Lingual Transfer with Meta Learningcited by 66
  38. 2019Induction Networks for Few-Shot Text Classificationcited by 64

50 most cited of 125 All 125 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 10; 2024: 6; 2025: 4; 2026: 3.

26 papers.

Sort:

By area:

  1. 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 409
  2. 2023AugGPT: Leveraging ChatGPT for Text Data Augmentationcited by 128
  3. 2023Mask-guided BERT for Few Shot Text Classificationcited by 25
  4. 2024Towards Robust Physical-world Backdoor Attacks on Lane Detectioncited by 25
  5. 2023Cross-city Few-Shot Traffic Forecasting via Traffic Pattern Bankcited by 22
  6. 2023A Survey on Stability of Learning with Limited Labelled Data and its Sensitivity to the Effects of Randomnesscited by 14
  7. 2022Federated Learning and Meta Learning: Approaches, Applications, and Directionscited by 12
  8. 2023Learning to Initialize: Can Meta Learning Improve Cross-task Generalization in Prompt Tuning?cited by 11
  9. 2025Meta-Thinking in LLMs via Multi-Agent Reinforcement Learning: A Surveycited by 10
  10. 2024Multi-scale Traffic Pattern Bank for Cross-city Few-shot Traffic Forecastingcited by 7
  11. 2024Towards Incremental Learning in Large Language Models: A Critical Reviewcited by 6
  12. 2023Hyperbolic Graph Neural Networks at Scale: A Meta Learning Approachcited by 5
14 more
  1. 2023MathNAS: If Blocks Have a Role in Mathematical Architecture Designcited by 4
  2. 2024MetaFollower: Adaptable Personalized Autonomous Car Followingcited by 4
  3. 2024CoLLEGe: Concept Embedding Generation for Large Language Modelscited by 3
  4. 2026STRUCTSURVEY: Structured Agentic Retrieval for Automated Survey Paper Generationcited by 3
  5. 2022The Devil is in the Details: On Models and Training Regimes for Few-Shot Intent Classificationcited by 2
  6. 2022m4Adapter: Multilingual Multi-Domain Adaptation for Machine Translation with a Meta-Adaptercited by 2
  7. 2023BERT Goes Off-Topic: Investigating the Domain Transfer Challenge using Genre Classificationcited by 2
  8. 2023Multilingual Sentence-Level Semantic Search using Meta-Distillation Learningcited by 1
  9. 2025XDoGE: Multilingual Data Reweighting to Enhance Language Inclusivity in LLMscited by 1
  10. 2026Structural priors for data-efficient language learningcited by 1
  11. 2024Adaptive Guidance for Local Training in Heterogeneous Federated Learningcited by 0
  12. 2025Latent Representation Learning of Multi-scale Thermophysics: Application to Dynamics in Shocked Porous Energetic Materialcited by 0
  13. 2025Agentic Meta-Orchestrator for Multi-task Copilotscited by 0
  14. 2026Meta-Learning and Meta-Reinforcement Learning -- Tracing the Path towards DeepMind's Adaptive Agentcited by 0