mlsanityData to 7 Oct 2026

Unsupervised Discovery of Clinical Disease Signatures Using Probabilistic Independence

arXiv 2402.05802 ↗First posted 8 Feb 2024cs.LG stat.AP stat.MLcited by 3

Not on the map. Why?

Abstract (arXiv)

Insufficiently precise diagnosis of clinical disease is likely responsible for many treatment failures, even for common conditions and treatments. With a large enough dataset, it may be possible to use unsupervised machine learning to define clinical disease patterns more precisely. We present an approach to learning these patterns by using probabilistic independence to disentangle the imprint on the medical record of causal latent sources of disease. We inferred a broad set of 2000 clinical signatures of latent sources from 9195 variables in 269,099 Electronic Health Records. The learned signatures produced better discrimination than the original variables in a lung cancer prediction task unknown to the inference algorithm, predicting 3-year malignancy in patients with no history of cancer before a solitary lung nodule was discovered. More importantly, the signatures' greater explanatory power identified pre-nodule signatures of apparently undiagnosed cancer in many of those patients.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 18 of its 71 references that are in the map. First posted on arXiv 8 Feb 2024.

Builds on

18 of 71 references.

References · 18

  1. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  2. 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 488
  3. 2016Tutorial on Variational Autoencoderscited by 404
  4. 2018Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learningcited by 278
  5. 2019Learning the Graphical Structure of Electronic Health Records with Graph Convolutional Transformercited by 88
  6. 2020Language Models Are An Effective Patient Representation Learning Technique For Electronic Health Record Datacited by 85
  7. 2023Nonlinear Independent Component Analysis for Principled Disentanglement in Unsupervised Deep Learningcited by 48
  8. 2020Deep Representation Learning of Electronic Health Records to Unlock Patient Stratification at Scalecited by 39
  9. 2022Identifying Patient-Specific Root Causes with the Heteroscedastic Noise Modelcited by 38
  10. 2019TAPER: Time-Aware Patient EHR Representationcited by 22
  11. 2015Distilling Knowledge from Deep Networks with Applications to Healthcare Domaincited by 16
  12. 2022Multimodal data matters: language model pre-training over structured and unstructured electronic health recordscited by 13
6 more
  1. 2019Representation Learning for Electronic Health Recordscited by 8
  2. 2022Identifying Patient-Specific Root Causes of Diseasecited by 8
  3. 2023Longitudinal Multimodal Transformer Integrating Imaging and Latent Clinical Signatures From Routine EHRs for Pulmonary Nodule Classificationcited by 8
  4. 2014Efficient Inference of Gaussian Process Modulated Renewal Processes with Application to Medical Event Datacited by 7
  5. 2022Sample-Specific Root Causal Inference with Latent Variablescited by 6
  6. 2021Generalized and Transferable Patient Language Representation for Phenotyping with Limited Datacited by 3

Built on by

Cited by year

Citing papers per year: 2025: 3.

3 papers.

  1. 2025Self-supervised learning of imaging and clinical signatures using a multimodal joint-embedding predictive architecturecited by 1
  2. 2025Cryptogenic stroke and migraine: using probabilistic independence and machine learning to uncover latent sources of disease from the electronic health recordcited by 0
  3. 2025A tutorial on discovering and quantifying the effect of latent causal sources of multimodal EHR datacited by 0