# Phenotyping of Clinical Notes with Improved Document Classification Models Using Contextualized Neural Language Models

arXiv: https://arxiv.org/abs/1910.13664 · HTML: https://mlsanity.com/paper/1910.13664/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Oct 2019.

## Built on by (most cited first, 20 of 20)

- [Revisiting Transformer-based Models for Long Document Classification](https://mlsanity.com/paper/2204.06683/) (2022)
- [On the Use of BERT for Automated Essay Scoring: Joint Learning of Multi-Scale Essay Representation](https://mlsanity.com/paper/2205.03835/) (2022)
- [Does the Magic of BERT Apply to Medical Code Assignment? A Quantitative Study](https://mlsanity.com/paper/2103.06511/) (2021)
- [CLIP: A Dataset for Extracting Action Items for Physicians from Hospital Discharge Notes](https://mlsanity.com/paper/2106.02524/) (2021)
- [Incorporating Uncertainty in Learning to Defer Algorithms for Safe Computer-Aided Diagnosis](https://mlsanity.com/paper/2108.07392/) (2021)
- [Finding the Law: Enhancing Statutory Article Retrieval via Graph Neural Networks](https://mlsanity.com/paper/2301.12847/) (2023)
- [Identifying and Characterizing Active Citizens who Refute Misinformation in Social Media](https://mlsanity.com/paper/2204.10080/) (2022)
- [Applying Automatic Text Summarization for Fake News Detection](https://mlsanity.com/paper/2204.01841/) (2022)
- [Which anonymization technique is best for which NLP task? -- It depends. A Systematic Study on Clinical Text Processing](https://mlsanity.com/paper/2209.00262/) (2022)
- [Text Guide: Improving the quality of long text classification by a text selection method based on feature importance](https://mlsanity.com/paper/2104.07225/) (2021)
- [Patient Outcome and Zero-shot Diagnosis Prediction with Hypernetwork-guided Multitask Learning](https://mlsanity.com/paper/2109.03062/) (2021)
- [Cross-Lingual Knowledge Transfer for Clinical Phenotyping](https://mlsanity.com/paper/2208.01912/) (2022)
- [Neural Language Models with Distant Supervision to Identify Major Depressive Disorder from Clinical Notes](https://mlsanity.com/paper/2104.09644/) (2021)
- [Improving VTE Identification through Adaptive NLP Model Selection and Clinical Expert Rule-based Classifier from Radiology Reports](https://mlsanity.com/paper/2309.12273/) (2023)
- [A BERT-Based Summarization approach for depression detection](https://mlsanity.com/paper/2409.08483/) (2024)
- [Controlling for Unobserved Confounding with Large Language Model Classification of Patient Smoking Status](https://mlsanity.com/paper/2411.03004/) (2024)
- [SkIn: Skimming-Intensive Long-Text Classification Using BERT for Medical Corpus](https://mlsanity.com/paper/2209.05741/) (2022)
- [Investigating Alternative Feature Extraction Pipelines For Clinical Note Phenotyping](https://mlsanity.com/paper/2310.03772/) (2023)
- [Improving VTE Identification through Language Models from Radiology Reports: A Comparative Study of Mamba, Phi-3 Mini, and BERT](https://mlsanity.com/paper/2408.09043/) (2024)
- [A Clinical Trial Design Approach to Auditing Language Models in Healthcare Setting](https://mlsanity.com/paper/2411.16702/) (2024)

## Abstract (arXiv)

Clinical notes contain an extensive record of a patient's health status, such as smoking status or the presence of heart conditions. However, this detail is not replicated within the structured data of electronic health systems. Phenotyping, the extraction of patient conditions from free clinical text, is a critical task which supports avariety of downstream applications such as decision support and secondary use of medical records. Previous work has resulted in systems which are high performing but require hand engineering, often of rules. Recent work in pretrained contextualized language models have enabled advances in representing text for a variety of tasks. We therefore explore several architectures for modeling pheno-typing that rely solely on BERT representations of the clinical note, removing the need for manual engineering. We find these architectures are competitive with or outperform existing state of the art methods on two phenotyping tasks.