# HealthPrompt: A Zero-shot Learning Paradigm for Clinical Natural Language Processing

arXiv: https://arxiv.org/abs/2203.05061 · HTML: https://mlsanity.com/paper/2203.05061/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 39 references that are in the map.
- First posted on arXiv 9 Mar 2022.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Publicly Available Clinical BERT Embeddings](https://mlsanity.com/paper/1904.03323/) (2019)
- [Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference](https://mlsanity.com/paper/2001.07676/) (2020)
- [Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Ugly](https://mlsanity.com/paper/1707.00600/) (2017)
- [Pre-trained Models for Natural Language Processing: A Survey](https://mlsanity.com/paper/2003.08271/) (2020)
- [Zero-Shot Learning Through Cross-Modal Transfer](https://mlsanity.com/paper/1301.3666/) (2013)
- [Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification](https://mlsanity.com/paper/2108.02035/) (2021)
- [Template-Based Named Entity Recognition Using BART](https://mlsanity.com/paper/2106.01760/) (2021)
- [COVID-Twitter-BERT: A Natural Language Processing Model to Analyse COVID-19 Content on Twitter](https://mlsanity.com/paper/2005.07503/) (2020)
- [Prompt-Learning for Fine-Grained Entity Typing](https://mlsanity.com/paper/2108.10604/) (2021)
- [Classifying Long Clinical Documents with Pre-trained Transformers](https://mlsanity.com/paper/2105.06752/) (2021)

## Built on by (most cited first, 26 of 26)

- [Foundation Model for Advancing Healthcare: Challenges, Opportunities, and Future Directions](https://mlsanity.com/paper/2404.03264/) (2024)
- [Prompt Engineering for Healthcare: Methodologies and Applications](https://mlsanity.com/paper/2304.14670/) (2023)
- [CPLLM: Clinical Prediction with Large Language Models](https://mlsanity.com/paper/2309.11295/) (2023)
- [Clinical Prompt Learning with Frozen Language Models](https://mlsanity.com/paper/2205.05535/) (2022)
- [An Empirical Evaluation of Prompting Strategies for Large Language Models in Zero-Shot Clinical Natural Language Processing](https://mlsanity.com/paper/2309.08008/) (2023)
- [Multi-label Few-shot ICD Coding as Autoregressive Generation with Prompt](https://mlsanity.com/paper/2211.13813/) (2022)
- [Prompt engineering paradigms for medical applications: scoping review and recommendations for better practices](https://mlsanity.com/paper/2405.01249/) (2024)
- [Large Language Models for Propaganda Detection](https://mlsanity.com/paper/2310.06422/) (2023)
- [A Survey on Responsible Generative AI: What to Generate and What Not](https://mlsanity.com/paper/2404.05783/) (2024)
- [Exploiting prompt learning with pre-trained language models for Alzheimer's Disease detection](https://mlsanity.com/paper/2210.16539/) (2022)
- [Leveraging Large Language Models to Extract Information on Substance Use Disorder Severity from Clinical Notes: A Zero-shot Learning Approach](https://mlsanity.com/paper/2403.12297/) (2024)
- [Mining Clinical Notes for Physical Rehabilitation Exercise Information: Natural Language Processing Algorithm Development and Validation Study](https://mlsanity.com/paper/2303.13466/) (2023)
- [Large Language Models for Healthcare Text Classification: A Systematic Review](https://mlsanity.com/paper/2503.01159/) (2025)
- [WC-SBERT: Zero-Shot Text Classification via SBERT with Self-Training for Wikipedia Categories](https://mlsanity.com/paper/2307.15293/) (2023)
- [Clinical information extraction for Low-resource languages with Few-shot learning using Pre-trained language models and Prompting](https://mlsanity.com/paper/2403.13369/) (2024)
- [Few-Shot Learning for Clinical Natural Language Processing Using Siamese Neural Networks](https://mlsanity.com/paper/2208.14923/) (2022)
- [Zero- and Few-shot Named Entity Recognition and Text Expansion in Medication Prescriptions using ChatGPT](https://mlsanity.com/paper/2409.17683/) (2024)
- [Leveraging Contextual Relatedness to Identify Suicide Documentation in Clinical Notes through Zero Shot Learning](https://mlsanity.com/paper/2301.03531/) (2023)
- [Eye-SpatialNet: Spatial Information Extraction from Ophthalmology Notes](https://mlsanity.com/paper/2305.11948/) (2023)
- [ConES: Concept Embedding Search for Parameter Efficient Tuning Large Vision Language Models](https://mlsanity.com/paper/2305.18993/) (2023)
- [A Question Answering Based Pipeline for Comprehensive Chinese EHR Information Extraction](https://mlsanity.com/paper/2402.11177/) (2024)
- [Effects of Different Prompts on the Quality of GPT-4 Responses to Dementia Care Questions](https://mlsanity.com/paper/2404.08674/) (2024)
- [Prompting Encoder Models for Zero-Shot Classification: A Cross-Domain Study in Italian](https://mlsanity.com/paper/2407.20654/) (2024)
- [Automating Adjudication of Cardiovascular Events Using Large Language Models](https://mlsanity.com/paper/2503.17222/) (2025)
- [Evaluating ChatGPT's Performance in Classifying Pneumonia from Chest X-Ray Images](https://mlsanity.com/paper/2510.21839/) (2025)
- [Multi-Modal Zero-Shot Prediction of Color Trajectories in Food Drying](https://mlsanity.com/paper/2512.06190/) (2025)

## Abstract (arXiv)

Deep learning algorithms are dependent on the availability of large-scale annotated clinical text datasets. The lack of such publicly available datasets is the biggest bottleneck for the development of clinical Natural Language Processing(NLP) systems. Zero-Shot Learning(ZSL) refers to the use of deep learning models to classify instances from new classes of which no training data have been seen before. Prompt-based learning is an emerging ZSL technique where we define task-based templates for NLP tasks. We developed a novel prompt-based clinical NLP framework called HealthPrompt and applied the paradigm of prompt-based learning on clinical texts. In this technique, rather than fine-tuning a Pre-trained Language Model(PLM), the task definitions are tuned by defining a prompt template. We performed an in-depth analysis of HealthPrompt on six different PLMs in a no-data setting. Our experiments prove that prompts effectively capture the context of clinical texts and perform remarkably well without any training data.