# Exploring deep learning methods for recognizing rare diseases and their clinical manifestations from texts

arXiv: https://arxiv.org/abs/2109.00343 · HTML: https://mlsanity.com/paper/2109.00343/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 78 references that are in the map.
- First posted on arXiv 1 Sep 2021.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Publicly Available Clinical BERT Embeddings](https://mlsanity.com/paper/1904.03323/) (2019)
- [Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation](https://mlsanity.com/paper/2010.16061/) (2020)
- [A Survey on Deep Learning for Named Entity Recognition](https://mlsanity.com/paper/1812.09449/) (2018)
- [Enriching Pre-trained Language Model with Entity Information for Relation Classification](https://mlsanity.com/paper/1905.08284/) (2019)
- [Text Classification Improved by Integrating Bidirectional LSTM with Two-dimensional Max Pooling](https://mlsanity.com/paper/1611.06639/) (2016)
- [The RareDis corpus: a corpus annotated with rare diseases, their signs and symptoms](https://mlsanity.com/paper/2108.01204/) (2021)

## Built on by (most cited first, 7 of 7)

- [Identifying and Extracting Rare Disease Phenotypes with Large Language Models](https://mlsanity.com/paper/2306.12656/) (2023)
- [Large Language Models Struggle in Token-Level Clinical Named Entity Recognition](https://mlsanity.com/paper/2407.00731/) (2024)
- [A Multi-granularity Concept Sparse Activation and Hierarchical Knowledge Graph Fusion Framework for Rare Disease Diagnosis](https://mlsanity.com/paper/2507.08529/) (2025)
- [RDMA: Cost Effective Agent-Driven Rare Disease Mining from Electronic Health Records](https://mlsanity.com/paper/2507.15867/) (2025)
- [Leveraging Large Language Models for Rare Disease Named Entity Recognition](https://mlsanity.com/paper/2508.09323/) (2025)
- [An artificial intelligence framework for end-to-end rare disease phenotyping from clinical notes using large language models](https://mlsanity.com/paper/2602.20324/) (2026)
- [Hindsight-Guided Rationale Distillation for Rare Disease Diagnosis](https://mlsanity.com/paper/2610.03176/) (2026)

## Abstract (arXiv)

Although rare diseases are characterized by low prevalence, approximately 300 million people are affected by a rare disease. The early and accurate diagnosis of these conditions is a major challenge for general practitioners, who do not have enough knowledge to identify them. In addition to this, rare diseases usually show a wide variety of manifestations, which might make the diagnosis even more difficult. A delayed diagnosis can negatively affect the patient's life. Therefore, there is an urgent need to increase the scientific and medical knowledge about rare diseases. Natural Language Processing (NLP) and Deep Learning can help to extract relevant information about rare diseases to facilitate their diagnosis and treatments. The paper explores the use of several deep learning techniques such as Bidirectional Long Short Term Memory (BiLSTM) networks or deep contextualized word representations based on Bidirectional Encoder Representations from Transformers (BERT) to recognize rare diseases and their clinical manifestations (signs and symptoms) in the RareDis corpus. This corpus contains more than 5,000 rare diseases and almost 6,000 clinical manifestations. BioBERT, a domain-specific language representation based on BERT and trained on biomedical corpora, obtains the best results. In particular, this model obtains an F1-score of 85.2% for rare diseases, outperforming all the other models.