# Few-shot learning for medical text: A systematic review

arXiv: https://arxiv.org/abs/2204.14081 · HTML: https://mlsanity.com/paper/2204.14081/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 97 references that are in the map.
- First posted on arXiv 21 Apr 2022.

## Builds on (most cited first, 30 of 31)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [Meta-Learning in Neural Networks: A Survey](https://mlsanity.com/paper/2004.05439/) (2020)
- [Generalizing from a Few Examples: A Survey on Few-Shot Learning](https://mlsanity.com/paper/1904.05046/) (2019)
- [A Survey of Data Augmentation Approaches for NLP](https://mlsanity.com/paper/2105.03075/) (2021)
- [Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces](https://mlsanity.com/paper/1805.10190/) (2018)
- [Recent Trends in Deep Learning Based Natural Language Processing](https://mlsanity.com/paper/1708.02709/) (2017)
- [FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation](https://mlsanity.com/paper/1810.10147/) (2018)
- [Revisiting Local Descriptor based Image-to-Class Measure for Few-shot Learning](https://mlsanity.com/paper/1903.12290/) (2019)
- [Large-Scale Multi-Label Text Classification on EU Legislation](https://mlsanity.com/paper/1906.02192/) (2019)
- [FewRel 2.0: Towards More Challenging Few-Shot Relation Classification](https://mlsanity.com/paper/1910.07124/) (2019)
- [Few-shot Slot Tagging with Collapsed Dependency Transfer and Label-enhanced Task-adaptive Projection Network](https://mlsanity.com/paper/2006.05702/) (2020)
- [Induction Networks for Few-Shot Text Classification](https://mlsanity.com/paper/1902.10482/) (2019)
- [Towards Few-Shot Fact-Checking via Perplexity](https://mlsanity.com/paper/2103.09535/) (2021)
- [Few-Shot NLG with Pre-Trained Language Model](https://mlsanity.com/paper/1904.09521/) (2019)
- [An Empirical Study on Large-Scale Multi-Label Text Classification Including Few and Zero-Shot Labels](https://mlsanity.com/paper/2010.01653/) (2020)
- [Few-Shot Named Entity Recognition: A Comprehensive Study](https://mlsanity.com/paper/2012.14978/) (2020)
- [Learning Algorithms for Active Learning](https://mlsanity.com/paper/1708.00088/) (2017)
- [Learning Reporting Dynamics during Breaking News for Rumour Detection in Social Media](https://mlsanity.com/paper/1610.07363/) (2016)
- [Meta-Learning for Few-Shot NMT Adaptation](https://mlsanity.com/paper/2004.02745/) (2020)
- [Few-shot Learning for Named Entity Recognition in Medical Text](https://mlsanity.com/paper/1811.05468/) (2018)
- [Multi-label Few/Zero-shot Learning with Knowledge Aggregated from Multiple Label Graphs](https://mlsanity.com/paper/2010.07459/) (2020)
- [Med7: a transferable clinical natural language processing model for electronic health records](https://mlsanity.com/paper/2003.01271/) (2020)
- [Towards one-shot learning for rare-word translation with external experts](https://mlsanity.com/paper/1809.03182/) (2018)
- [FewJoint: A Few-shot Learning Benchmark for Joint Language Understanding](https://mlsanity.com/paper/2009.08138/) (2020)

## Built on by (most cited first, 13 of 13)

- [AugGPT: Leveraging ChatGPT for Text Data Augmentation](https://mlsanity.com/paper/2302.13007/) (2023)
- [Language Models for Text Classification: Is In-Context Learning Enough?](https://mlsanity.com/paper/2403.17661/) (2024)
- [Mask-guided BERT for Few Shot Text Classification](https://mlsanity.com/paper/2302.10447/) (2023)
- [MedHallBench: A New Benchmark for Assessing Hallucination in Medical Large Language Models](https://mlsanity.com/paper/2412.18947/) (2024)
- [Enhancing Small Medical Learners with Privacy-preserving Contextual Prompting](https://mlsanity.com/paper/2305.12723/) (2023)
- [Pretraining Decision Transformers with Reward Prediction for In-Context Multi-task Structured Bandit Learning](https://mlsanity.com/paper/2406.05064/) (2024)
- [Few Shot Learning for Medical Imaging: A Comparative Analysis of Methodologies and Formal Mathematical Framework](https://mlsanity.com/paper/2305.04401/) (2023)
- [ClinicRealm: Re-evaluating Large Language Models with Conventional Machine Learning for Non-Generative Clinical Prediction Tasks](https://mlsanity.com/paper/2407.18525/) (2024)
- [A Survey of Few-Shot Learning for Biomedical Time Series](https://mlsanity.com/paper/2405.02485/) (2024)
- [FCert: Certifiably Robust Few-Shot Classification in the Era of Foundation Models](https://mlsanity.com/paper/2404.08631/) (2024)
- [Few-Shot Learning for Clinical Natural Language Processing Using Siamese Neural Networks](https://mlsanity.com/paper/2208.14923/) (2022)
- [Extracting Biomedical Entities from Noisy Audio Transcripts](https://mlsanity.com/paper/2403.17363/) (2024)
- [Meta-learning Pathologies from Radiology Reports using Variance Aware Prototypical Networks](https://mlsanity.com/paper/2210.13979/) (2022)

## Abstract (arXiv)

Objective: Few-shot learning (FSL) methods require small numbers of labeled instances for training. As many medical topics have limited annotated textual data in practical settings, FSL-based natural language processing (NLP) methods hold substantial promise. We aimed to conduct a systematic review to explore the state of FSL methods for medical NLP. Materials and Methods: We searched for articles published between January 2016 and August 2021 using PubMed/Medline, Embase, ACL Anthology, and IEEE Xplore Digital Library. To identify the latest relevant methods, we also searched other sources such as preprint servers (eg., medRxiv) via Google Scholar. We included all articles that involved FSL and any type of medical text. We abstracted articles based on data source(s), aim(s), training set size(s), primary method(s)/approach(es), and evaluation method(s). Results: 31 studies met our inclusion criteria-all published after 2018; 22 (71%) since 2020. Concept extraction/named entity recognition was the most frequently addressed task (13/31; 42%), followed by text classification (10/31; 32%). Twenty-one (68%) studies reconstructed existing datasets to create few-shot scenarios synthetically, and MIMIC-III was the most frequently used dataset (7/31; 23%). Common methods included FSL with attention mechanisms (12/31; 39%), prototypical networks (8/31; 26%), and meta-learning (6/31; 19%). Discussion: Despite the potential for FSL in biomedical NLP, progress has been limited compared to domain-independent FSL. This may be due to the paucity of standardized, public datasets, and the relative underperformance of FSL methods on biomedical topics. Creation and release of specialized datasets for biomedical FSL may aid method development by enabling comparative analyses.