# Deep learning models are not robust against noise in clinical text

arXiv: https://arxiv.org/abs/2108.12242 · HTML: https://mlsanity.com/paper/2108.12242/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 32 references that are in the map.
- First posted on arXiv 27 Aug 2021.

## Builds on (most cited first, 12 of 12)

- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020)
- [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://mlsanity.com/paper/2005.04118/) (2020)
- [Publicly Available Clinical BERT Embeddings](https://mlsanity.com/paper/1904.03323/) (2019)
- [Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets](https://mlsanity.com/paper/1906.05474/) (2019)
- [Lessons from Natural Language Inference in the Clinical Domain](https://mlsanity.com/paper/1808.06752/) (2018)
- [Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis](https://mlsanity.com/paper/1706.03446/) (2017)
- [OpenAttack: An Open-source Textual Adversarial Attack Toolkit](https://mlsanity.com/paper/2009.09191/) (2020)
- [A Corpus with Multi-Level Annotations of Patients, Interventions and Outcomes to Support Language Processing for Medical Literature](https://mlsanity.com/paper/1806.04185/) (2018)
- [Deep Learning in Bioinformatics](https://mlsanity.com/paper/1603.06430/) (2016)
- [Clinical XLNet: Modeling Sequential Clinical Notes and Predicting Prolonged Mechanical Ventilation](https://mlsanity.com/paper/1912.11975/) (2019)
- [Relation extraction from clinical texts using domain invariant convolutional neural network](https://mlsanity.com/paper/1606.09370/) (2016)

## Built on by (most cited first, 6 of 6)

- [Improving the robustness and accuracy of biomedical language models through adversarial training](https://mlsanity.com/paper/2111.08529/) (2021)
- [Forget-MI: Machine Unlearning for Forgetting Multimodal Information in Healthcare Settings](https://mlsanity.com/paper/2506.23145/) (2025)
- [Out-of-Distribution Generalization in Text Classification: Past, Present, and Future](https://mlsanity.com/paper/2305.14104/) (2023)
- [DeepProphet2 -- A Deep Learning Gene Recommendation Engine](https://mlsanity.com/paper/2208.01918/) (2022)
- [Improving ICD coding using Chapter based Named Entities and Attentional Models](https://mlsanity.com/paper/2407.17230/) (2024)
- [Towards Robust and Fair Next Visit Diagnosis Prediction under Noisy Clinical Notes with Large Language Models](https://mlsanity.com/paper/2511.18393/) (2025)

## Abstract (arXiv)

Artificial Intelligence (AI) systems are attracting increasing interest in the medical domain due to their ability to learn complicated tasks that require human intelligence and expert knowledge. AI systems that utilize high-performance Natural Language Processing (NLP) models have achieved state-of-the-art results on a wide variety of clinical text processing benchmarks. They have even outperformed human accuracy on some tasks. However, performance evaluation of such AI systems have been limited to accuracy measures on curated and clean benchmark datasets that may not properly reflect how robustly these systems can operate in real-world situations. In order to address this challenge, we introduce and implement a wide variety of perturbation methods that simulate different types of noise and variability in clinical text data. While noisy samples produced by these perturbation methods can often be understood by humans, they may cause AI systems to make erroneous decisions. Conducting extensive experiments on several clinical text processing tasks, we evaluated the robustness of high-performance NLP models against various types of character-level and word-level noise. The results revealed that the NLP models performance degrades when the input contains small amounts of noise. This study is a significant step towards exposing vulnerabilities of AI models utilized in clinical text processing systems. The proposed perturbation methods can be used in performance evaluation tests to assess how robustly clinical NLP models can operate on noisy data, in real-world settings.