mlsanityData to 7 Oct 2026

Multimodal Training to Unimodal Deployment: Leveraging Unstructured Data During Training to Optimize Structured Data Only Deployment

arXiv 2603.22530 ↗First posted 23 Mar 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Unstructured Electronic Health Record (EHR) data, such as clinical notes, contain clinical contextual observations that are not directly reflected in structured data fields. This additional information can substantially improve model learning. However, due to their unstructured nature, these data are often unavailable or impractical to use when deploying a model. We introduce a multimodal learning framework that leverages unstructured EHR data during training while producing a model that can be deployed using only structured EHR data. Using a cohort of 3,466 children evaluated for late talking, we generated note embeddings with BioClinicalBERT and encoded structured embeddings from demographics and medical codes. A note-based teacher model and a structured-only student model were jointly trained using contrastive learning and contrastive knowledge distillation loss, producing a strong classifier (AUROC = 0.985). Our proposed model reached an AUROC of 0.705, outperforming the structured-only baseline of 0.656. These results demonstrate that incorporating unstructured data during training enhances the model's capacity to identify task-relevant information within structured EHR data, enabling a deployable structured-only phenotype model.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 16 of its 38 references that are in the map. First posted on arXiv 23 Mar 2026.

Builds on

16 of 38 references.

References · 16

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  4. 2020Knowledge Distillation: A Surveycited by 1,578
  5. 2019Publicly Available Clinical BERT Embeddingscited by 995
  6. 2022MedCLIP: Contrastive Learning from Unpaired Medical Images and Textcited by 564
  7. 2020Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease predictioncited by 368
  8. 2018Explainable Prediction of Medical Codes from Clinical Textcited by 231
  9. 2024Deep Multimodal Learning with Missing Modality: A Surveycited by 89
  10. 2020Deep Partial Multi-View Learningcited by 71
  11. 2018Joint Embedding of Words and Labels for Text Classificationcited by 69
  12. 2020Deep Representation Learning of Patient Data from Electronic Health Records (EHR): A Systematic Reviewcited by 46
4 more
  1. 2023Learnable Cross-modal Knowledge Distillation for Multi-modal Learning with Missing Modalitycited by 44
  2. 2022CL4CTR: A Contrastive Learning Framework for CTR Predictioncited by 14
  3. 2021Machine Learning for Multimodal Electronic Health Records-based Research: Challenges and Perspectivescited by 7
  4. 2024CKD: Contrastive Knowledge Distillation from A Sample-wise Perspectivecited by 6

Built on by

Not cited yet.