mlsanityData to 14 Oct 2026

MedRep: Medical Concept Representation for General Electronic Health Record Foundation Models

arXiv 2504.08329 ↗First posted 11 Apr 2025cs.AI cs.CL cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Electronic health record (EHR) foundation models have been an area ripe for exploration with their improved performance in various medical tasks. Despite the rapid advances, there exists a fundamental limitation: Processing unseen medical codes out of vocabulary. This problem limits the generalizability of EHR foundation models and the integration of models trained with different vocabularies. To alleviate this problem, we propose a set of novel medical concept representations (MedRep) for EHR foundation models based on the observational medical outcome partnership (OMOP) common data model (CDM). For concept representation learning, we enrich the information of each concept with a minimal definition through large language model (LLM) prompts and complement the text-based representations through the graph ontology of OMOP vocabulary. Our approach outperforms the vanilla EHR foundation model and the model with a previously introduced medical code tokenizer in diverse prediction tasks. We also demonstrate the generalizability of MedRep through external validation.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 23 of its 60 references that are in the map. First posted on arXiv 11 Apr 2025.

Builds on

23 of 60 references.

References · 23

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  3. 2020Scaling Laws for Neural Language Modelscited by 6,586
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  5. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  6. 2016Learning without Forgettingcited by 2,496
  7. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  8. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 938
  9. 2020Deep Graph Contrastive Representation Learningcited by 539
  10. 2020Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease predictioncited by 373
  11. 2016RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanismcited by 349
  12. 2017Multitask learning and benchmarking with clinical time series datacited by 290
11 more
  1. 2019BEHRT: Transformer for Electronic Health Recordscited by 263
  2. 2019Clinically Accurate Chest X-Ray Report Generationcited by 139
  3. 2019Pre-training of Graph Augmented Transformers for Medication Recommendationcited by 127
  4. 2023EHRSHOT: An EHR Benchmark for Few-Shot Evaluation of Foundation Modelscited by 105
  5. 2024Zero Shot Health Trajectory Prediction Using Transformercited by 77
  6. 2021CEHR-BERT: Incorporating temporal information from structured EHR data to improve prediction taskscited by 76
  7. 2021Hi-BEHRT: Hierarchical Transformer-based model for accurate prediction of clinical events using multimodal longitudinal electronic health recordscited by 54
  8. 2023A Multi-Center Study on the Adaptability of a Shared Foundation Model for Electronic Health Recordscited by 31
  9. 2024MEIT: Multimodal Electrocardiogram Instruction Tuning on Large Language Models for Report Generationcited by 29
  10. 2025Multimodal Medical Code Tokenizercited by 14
  11. 2024Multi-task Heterogeneous Graph Learning on Electronic Health Recordscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2025Count-Based Approaches Remain Strong: A Benchmark Against Transformer and LLM Pipelines on Structured EHRcited by 3
  2. 2026A General Multimodal Probing Framework for Data Fusion and Prediction with Frozen Large Language Models: Evidence from Multimodal EHR Datacited by 0
  3. 2026GLoC-EHR: Evidence-Cited Clinical Reasoning over Global Context and Local EHR Eventscited by 0
  4. 2026Bridging the EHR Divide: Asymmetric Contrastive Learning for Cross-National Medical Representation Transfercited by 0