# Vocabulary Transfer for Biomedical Texts: Add Tokens if You Can Not Add Data

arXiv: https://arxiv.org/abs/2208.02554 · HTML: https://mlsanity.com/paper/2208.02554/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 15 references that are in the map.
- First posted on arXiv 4 Aug 2022.

## Builds on (most cited first, 5 of 5)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020)
- [ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission](https://mlsanity.com/paper/1904.05342/) (2019)
- [Medical Text Classification using Convolutional Neural Networks](https://mlsanity.com/paper/1704.06841/) (2017)

## Built on by (most cited first, 2 of 2)

- [Token Distillation: Attention-aware Input Embeddings For New Tokens](https://mlsanity.com/paper/2505.20133/) (2025)
- [Specialised or Generic? Tokenization Choices for Radiology Language Models](https://mlsanity.com/paper/2508.09952/) (2025)

## Abstract (arXiv)

Working within specific NLP subdomains presents significant challenges, primarily due to a persistent deficit of data. Stringent privacy concerns and limited data accessibility often drive this shortage. Additionally, the medical domain demands high accuracy, where even marginal improvements in model performance can have profound impacts. In this study, we investigate the potential of vocabulary transfer to enhance model performance in biomedical NLP tasks. Specifically, we focus on vocabulary extension, a technique that involves expanding the target vocabulary to incorporate domain-specific biomedical terms. Our findings demonstrate that vocabulary extension, leads to measurable improvements in both downstream model performance and inference time.