mlsanityData to 7 Oct 2026

Hierarchical Pretraining on Multimodal Electronic Health Records

arXiv 2310.07871 ↗First posted 11 Oct 2023cs.AIcited by 11

Not on the map. Why?

Abstract (arXiv)

Pretraining has proven to be a powerful technique in natural language processing (NLP), exhibiting remarkable success in various NLP downstream tasks. However, in the medical domain, existing pretrained models on electronic health records (EHR) fail to capture the hierarchical nature of EHR data, limiting their generalization capability across diverse downstream tasks using a single pretrained model. To tackle this challenge, this paper introduces a novel, general, and unified pretraining framework called MEDHMP, specifically designed for hierarchically multimodal EHR data. The effectiveness of the proposed MEDHMP is demonstrated through experimental results on eight downstream tasks spanning three levels. Comparisons against eighteen baselines further highlight the efficacy of our approach.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 28 of its 54 references that are in the map. First posted on arXiv 11 Oct 2023.

Builds on

28 of 54 references.

References · 28

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  5. 2019Publicly Available Clinical BERT Embeddingscited by 995
  6. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 531
  7. 2019ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmissioncited by 487
  8. 2015Unsupervised Learning of Video Representations using LSTMscited by 434
  9. 2020Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease predictioncited by 368
  10. 2016RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanismcited by 344
  11. 2019Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasetscited by 298
  12. 2019BEHRT: Transformer for Electronic Health Recordscited by 258
16 more
  1. 2019Pre-training of Graph Augmented Transformers for Medication Recommendationcited by 124
  2. 2017Dipole: Diagnosis Prediction in Healthcare via Attention-based Bidirectional Recurrent Neural Networkscited by 114
  3. 2021Self-Supervised Transformer for Sparse and Irregularly Sampled Multivariate Clinical Time-Seriescited by 96
  4. 2022BioBART: Pretraining and Evaluation of A Biomedical Generative Language Modelcited by 83
  5. 2018MiME: Multilevel Medical Embedding of Electronic Health Records for Predictive Healthcarecited by 72
  6. 2016Multi-layer Representation Learning for Medical Conceptscited by 68
  7. 2021MMBERT: Multimodal BERT Pretraining for Improved Medical VQAcited by 68
  8. 2023Best of Both Worlds: Multimodal Contrastive Learning with Tabular and Imaging Datacited by 68
  9. 2022Clinical-Longformer and Clinical-BigBird: Transformers for long clinical sequencescited by 67
  10. 2022Mixing Up Contrastive Learning: Self-Supervised Representation Learning for Time Seriescited by 54
  11. 2019AdaCare: Explainable Clinical Health Status Representation Learning via Scale-Adaptive Feature Extraction and Recalibrationcited by 50
  12. 2018RAIM: Recurrent Attentive and Intensive Model of Multimodal Patient Monitoring Datacited by 47
  13. 2019Probing Biomedical Embeddings from Language Modelscited by 39
  14. 2021How to Leverage Multimodal EHR Data for Better Medical Predictions?cited by 26
  15. 2022Multimodal data matters: language model pre-training over structured and unstructured electronic health recordscited by 13
  16. 2020UNITE: Uncertainty-based Health Risk Prediction Leveraging Multi-sourced Datacited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 6; 2026: 1.

11 papers.

  1. 2025Multi-modal Time Series Analysis: A Tutorial and Surveycited by 24
  2. 2024Recent Advances in Predictive Modeling with Electronic Health Recordscited by 11
  3. 2025OctoMed: Data Recipes for State-of-the-Art Multimodal Medical Reasoningcited by 10
  4. 2024FEDMEKI: A Benchmark for Scaling Medical Foundation Models via Federated Knowledge Injectioncited by 8
  5. 2024Synthesizing Multimodal Electronic Health Records via Predictive Diffusion Modelscited by 7
  6. 2025Memorize and Rank: Elevating Large Language Models for Clinical Diagnosis Predictioncited by 6
  7. 2024Contrastive Learning on Multimodal Analysis of Electronic Health Recordscited by 2
  8. 2025MEDMKG: Benchmarking Medical Knowledge Exploitation with Multimodal Knowledge Graphcited by 2
  9. 2025Early GVHD Prediction in Liver Transplantation via Multi-Modal Deep Learning on Imbalanced EHR Datacited by 0
  10. 2025Patient-level Information Extraction by Consistent Integration of Textual and Tabular Evidence with Bayesian Networkscited by 0
  11. 2026A Clinical Point Cloud Paradigm for In-Hospital Mortality Prediction from Multi-Level Incomplete Multimodal EHRscited by 0