# Automatic Assignment of Radiology Examination Protocols Using Pre-trained Language Models with Knowledge Distillation

arXiv: https://arxiv.org/abs/2009.00694 · HTML: https://mlsanity.com/paper/2009.00694/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Sep 2020.

## Built on by (most cited first, 1 of 1)

- [Building blocks for complex tasks: Robust generative event extraction for radiology reports under domain shifts](https://mlsanity.com/paper/2306.09544/) (2023)

## Abstract (arXiv)

Selecting radiology examination protocol is a repetitive, and time-consuming process. In this paper, we present a deep learning approach to automatically assign protocols to computer tomography examinations, by pre-training a domain-specific BERT model (BERTrad). To handle the high data imbalance across exam protocols, we used a knowledge distillation approach that up-sampled the minority classes through data augmentation. We compared classification performance of the described approach with the statistical n-gram models using Support Vector Machine (SVM), Gradient Boosting Machine (GBM), and Random Forest (RF) classifiers, as well as the Google's BERTbase model. SVM, GBM and RF achieved macro-averaged F1 scores of 0.45, 0.45, and 0.6 while BERTbase and BERTrad achieved 0.61 and 0.63. Knowledge distillation improved overall performance on the minority classes, achieving a F1 score of 0.66.