# Rare Disease Identification from Clinical Notes with Ontologies and Weak Supervision

arXiv: https://arxiv.org/abs/2105.01995 · HTML: https://mlsanity.com/paper/2105.01995/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 28 references that are in the map.
- First posted on arXiv 5 May 2021.

## Builds on (most cited first, 6 of 6)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets](https://mlsanity.com/paper/1906.05474/) (2019)
- [Explainable Automated Coding of Clinical Notes using Hierarchical Label-wise Attention Networks and Label Embedding Initialisation](https://mlsanity.com/paper/2010.15728/) (2020)
- [Experimental Evaluation and Development of a Silver-Standard for the MIMIC-III Clinical Coding Dataset](https://mlsanity.com/paper/2006.07332/) (2020)
- [Bio-YODIE: A Named Entity Linking System for Biomedical Text](https://mlsanity.com/paper/1811.04860/) (2018)
- [Extracting Lifestyle Factors for Alzheimer's Disease from Clinical Notes Using Deep Learning with Weak Supervision](https://mlsanity.com/paper/2101.09244/) (2021)

## Built on by (most cited first, 5 of 5)

- [Automated Clinical Coding: What, Why, and Where We Are?](https://mlsanity.com/paper/2203.11092/) (2022)
- [Ontology-Driven and Weakly Supervised Rare Disease Identification from Clinical Notes](https://mlsanity.com/paper/2205.05656/) (2022)
- [Surpassing GPT-4 Medical Coding with a Two-Stage Approach](https://mlsanity.com/paper/2311.13735/) (2023)
- [Towards Semi-Structured Automatic ICD Coding via Tree-based Contrastive Learning](https://mlsanity.com/paper/2310.09672/) (2023)
- [RuCCoD: Towards Automated ICD Coding in Russian](https://mlsanity.com/paper/2502.21263/) (2025)

## Abstract (arXiv)

The identification of rare diseases from clinical notes with Natural Language Processing (NLP) is challenging due to the few cases available for machine learning and the need of data annotation from clinical experts. We propose a method using ontologies and weak supervision. The approach includes two steps: (i) Text-to-UMLS, linking text mentions to concepts in Unified Medical Language System (UMLS), with a named entity linking tool (e.g. SemEHR) and weak supervision based on customised rules and Bidirectional Encoder Representations from Transformers (BERT) based contextual representations, and (ii) UMLS-to-ORDO, matching UMLS concepts to rare diseases in Orphanet Rare Disease Ontology (ORDO). Using MIMIC-III US intensive care discharge summaries as a case study, we show that the Text-to-UMLS process can be greatly improved with weak supervision, without any annotated data from domain experts. Our analysis shows that the overall pipeline processing discharge summaries can surface rare disease cases, which are mostly uncaptured in manual ICD codes of the hospital admissions.