# Unsupervised Machine Learning for the Discovery of Latent Disease Clusters and Patient Subgroups Using Electronic Health Records

arXiv: https://arxiv.org/abs/1905.10309 · HTML: https://mlsanity.com/paper/1905.10309/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 May 2019.

## Built on by (most cited first, 11 of 11)

- [Cluster Explanation via Polyhedral Descriptions](https://mlsanity.com/paper/2210.08798/) (2022)
- [UCSL : A Machine Learning Expectation-Maximization framework for Unsupervised Clustering driven by Supervised Learning](https://mlsanity.com/paper/2107.01988/) (2021)
- [Mining Electronic Health Records to Investigate Effectiveness of Ensemble Deep Clustering](https://mlsanity.com/paper/2604.07085/) (2026)
- [SubgroupTE: Advancing Treatment Effect Estimation with Subgroup Identification](https://mlsanity.com/paper/2401.12369/) (2024)
- [Enabling scalable clinical interpretation of ML-based phenotypes using real world data](https://mlsanity.com/paper/2208.01607/) (2022)
- [Introduction to Machine Learning for Physicians: A Survival Guide for Data Deluge](https://mlsanity.com/paper/2212.12303/) (2022)
- [Medical Pathologies Prediction : Systematic Review and Proposed Approach](https://mlsanity.com/paper/2304.00311/) (2023)
- [Machine Learning Applications In Healthcare: The State Of Knowledge and Future Directions](https://mlsanity.com/paper/2307.14067/) (2023)
- [Sequential Inference of Hospitalization Electronic Health Records Using Probabilistic Models](https://mlsanity.com/paper/2403.19011/) (2024)
- [Risk Factor Identification In Osteoporosis Using Unsupervised Machine Learning Techniques](https://mlsanity.com/paper/2405.15882/) (2024)
- [Fair Clustering with Minimum Representation Constraints](https://mlsanity.com/paper/2409.02963/) (2024)

## Abstract (arXiv)

Machine learning has become ubiquitous and a key technology on mining electronic health records (EHRs) for facilitating clinical research and practice. Unsupervised machine learning, as opposed to supervised learning, has shown promise in identifying novel patterns and relations from EHRs without using human created labels. In this paper, we investigate the application of unsupervised machine learning models in discovering latent disease clusters and patient subgroups based on EHRs. We utilized Latent Dirichlet Allocation (LDA), a generative probabilistic model, and proposed a novel model named Poisson Dirichlet Model (PDM), which extends the LDA approach using a Poisson distribution to model patients' disease diagnoses and to alleviate age and sex factors by considering both observed and expected observations. In the empirical experiments, we evaluated LDA and PDM on three patient cohorts with EHR data retrieved from the Rochester Epidemiology Project (REP), for the discovery of latent disease clusters and patient subgroups. We compared the effectiveness of LDA and PDM in identifying latent disease clusters through the visualization of disease representations learned by two approaches. We also tested the performance of LDA and PDM in differentiating patient subgroups through survival analysis, as well as statistical analysis. The experimental results show that the proposed PDM could effectively identify distinguished disease clusters by alleviating the impact of age and sex, and that LDA could stratify patients into more differentiable subgroups than PDM in terms of p-values. However, the subgroups discovered by PDM might imply the underlying patterns of diseases of greater interest in epidemiology research due to the alleviation of age and sex. Both unsupervised machine learning approaches could be leveraged to discover patient subgroups using EHRs but with different foci.