# Zero-Shot and Few-Shot Classification of Biomedical Articles in Context of the COVID-19 Pandemic

arXiv: https://arxiv.org/abs/2201.03017 · HTML: https://mlsanity.com/paper/2201.03017/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 21 references that are in the map.
- First posted on arXiv 9 Jan 2022.

## Builds on (most cited first, 10 of 10)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [What you can cram into a single vector: Probing sentence embeddings for linguistic properties](https://mlsanity.com/paper/1805.01070/) (2018)
- [What do you learn from context? Probing for sentence structure in contextualized word representations](https://mlsanity.com/paper/1905.06316/) (2019)
- [An Empirical Study on Large-Scale Multi-Label Text Classification Including Few and Zero-Shot Labels](https://mlsanity.com/paper/2010.01653/) (2020)
- [Knowledge-aware Zero-Shot Learning: Survey and Perspective](https://mlsanity.com/paper/2103.00070/) (2021)
- [Probing Biomedical Embeddings from Language Models](https://mlsanity.com/paper/1904.02181/) (2019)
- [Probing Pre-Trained Language Models for Disease Knowledge](https://mlsanity.com/paper/2106.07285/) (2021)
- [Metric Learning for Dynamic Text Classification](https://mlsanity.com/paper/1911.01026/) (2019)

## Built on by (most cited first, 1 of 1)

- [MedAI Dialog Corpus (MEDIC): Zero-Shot Classification of Doctor and AI Responses in Health Consultations](https://mlsanity.com/paper/2310.12489/) (2023)

## Abstract (arXiv)

MeSH (Medical Subject Headings) is a large thesaurus created by the National Library of Medicine and used for fine-grained indexing of publications in the biomedical domain. In the context of the COVID-19 pandemic, MeSH descriptors have emerged in relation to articles published on the corresponding topic. Zero-shot classification is an adequate response for timely labeling of the stream of papers with MeSH categories. In this work, we hypothesise that rich semantic information available in MeSH has potential to improve BioBERT representations and make them more suitable for zero-shot/few-shot tasks. We frame the problem as determining if MeSH term definitions, concatenated with paper abstracts are valid instances or not, and leverage multi-task learning to induce the MeSH hierarchy in the representations thanks to a seq2seq task. Results establish a baseline on the MedLine and LitCovid datasets, and probing shows that the resulting representations convey the hierarchical relations present in MeSH.