mlsanityData to 7 Oct 2026

Unifying Heterogeneous Electronic Health Records Systems via Text-Based Code Embedding

arXiv 2108.03625 ↗First posted 8 Aug 2021cs.LG cs.NEcited by 24

Not on the map. Why?

Abstract (arXiv)

Substantial increase in the use of Electronic Health Records (EHRs) has opened new frontiers for predictive healthcare. However, while EHR systems are nearly ubiquitous, they lack a unified code system for representing medical concepts. Heterogeneous formats of EHR present a substantial barrier for the training and deployment of state-of-the-art deep learning models at scale. To overcome this problem, we introduce Description-based Embedding, DescEmb, a code-agnostic description-based representation learning framework for predictive modeling on EHR. DescEmb takes advantage of the flexibility of neural language understanding models while maintaining a neutral approach that can be combined with prior frameworks for task-specific representation learning or predictive modeling. We tested our model's capacity on various experiments including prediction tasks, transfer learning and pooled learning. DescEmb shows higher performance in overall experiments compared to code-based approach, opening the door to a text-based approach in predictive healthcare research that is not constrained by EHR structure nor special domain knowledge.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 28 of its 45 references that are in the map. First posted on arXiv 8 Aug 2021.

Builds on

28 of 45 references.

References · 28

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  5. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  6. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  7. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  8. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
  9. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 996
  10. 2019Publicly Available Clinical BERT Embeddingscited by 995
  11. 2020Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease predictioncited by 368
  12. 2016RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanismcited by 344
16 more
  1. 2019Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasetscited by 298
  2. 2018Scalable and accurate deep learning for electronic health recordscited by 275
  3. 2019BEHRT: Transformer for Electronic Health Recordscited by 258
  4. 2015Doctor AI: Predicting Clinical Events via Recurrent Neural Networkscited by 205
  5. 2016GRAM: Graph-based Attention Model for Healthcare Representation Learningcited by 152
  6. 2019Pre-training of Graph Augmented Transformers for Medication Recommendationcited by 124
  7. 2017Dipole: Diagnosis Prediction in Healthcare via Attention-based Bidirectional Recurrent Neural Networkscited by 114
  8. 2017Attend and Diagnose: Clinical Time Series Analysis using Attention Modelscited by 105
  9. 2015Learning to Diagnose with LSTM Recurrent Neural Networkscited by 99
  10. 2019Learning the Graphical Structure of Electronic Health Records with Graph Convolutional Transformercited by 88
  11. 2020Language Models Are An Effective Patient Representation Learning Technique For Electronic Health Record Datacited by 85
  12. 2018MiME: Multilevel Medical Embedding of Electronic Health Records for Predictive Healthcarecited by 72
  13. 2016Multi-layer Representation Learning for Medical Conceptscited by 68
  14. 2016Deepr: A Convolutional Net for Medical Recordscited by 59
  15. 2020Deep Representation Learning of Electronic Health Records to Unlock Patient Stratification at Scalecited by 39
  16. 2016DeepCare: A Deep Dynamic Memory Model for Predictive Medicinecited by 32

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 5; 2024: 2; 2025: 7; 2026: 6.

24 papers.

Sort:

By area:

  1. 2022GenHPF: General Healthcare Predictive Framework with Multi-task Multi-source Learningcited by 35
  2. 2023A Multi-Center Study on the Adaptability of a Shared Foundation Model for Electronic Health Recordscited by 30
  3. 2023The Shaky Foundations of Clinical Foundation Models: A Survey of Large Language Models and Foundation Models for EMRscited by 29
  4. 2023MediTab: Scaling Medical Tabular Data Predictors via Data Consolidation, Enrichment, and Refinementcited by 24
  5. 2023Multimodal Clinical Benchmark for Emergency Care (MC-BEC): A Comprehensive Benchmark for Evaluating Foundation Models in Emergency Medicinecited by 24
  6. 2024Emergency Department Decision Support using Clinical Pseudo-notescited by 22
  7. 2023General-Purpose Retrieval-Enhanced Medical Prediction Model Using Near-Infinite Historycited by 12
  8. 2024ACES: Automatic Cohort Extraction System for Event-Stream Datasetscited by 8
  9. 2025Representation learning to advance multi-institutional studies with electronic health record data from US and Francecited by 5
  10. 2025LabTOP: A Unified Model for Lab Test Outcome Prediction on Electronic Health Recordscited by 5
  11. 2022UniHPF : Universal Healthcare Predictive Framework with Zero Domain Knowledgecited by 4
  12. 2022sEHR-CE: Language modelling of structured EHR data for efficient and generalizable patient cohort expansioncited by 3
12 more
  1. 2025Time-Aware Attention for Enhanced Electronic Health Records Modelingcited by 3
  2. 2026PORTER: Language-Grounded Event Representations for Portable Structured EHR Foundation Modelscited by 2
  3. 2022Universal EHR Federated Learning Frameworkcited by 1
  4. 2026PaReGTA: A Temporally Aware LLM-Based Patient Representation Framework for EHR Analyticscited by 1
  5. 2026Multi-lingual Multi-institutional Electronic Health Record based Predictive Modelcited by 1
  6. 2025MD-ViSCo: A Unified Model for Multi-Directional Vital Sign Waveform Conversioncited by 0
  7. 2025OrdShap: Feature Position Importance for Sequential Black-Box Modelscited by 0
  8. 2025CURE: Controlled Unlearning for Robust Embeddings -- Mitigating Conceptual Shortcuts in Pre-Trained Language Modelscited by 0
  9. 2025EMR-AGENT: Automating Cohort and Feature Extraction from EMR Databasescited by 0
  10. 2026Imputation of Unknown Missingness in Sparse Electronic Health Recordscited by 0
  11. 2026Representation Before Training: A Practical Benchmark for Generative Medical Event Model Tokenizationcited by 0
  12. 2026EHR-RAGp: Retrieval-Augmented Prototype-Guided Foundation Model for Electronic Health Recordscited by 0