# Emergency Department Decision Support using Clinical Pseudo-notes

arXiv: https://arxiv.org/abs/2402.00160 · HTML: https://mlsanity.com/paper/2402.00160/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 55 references that are in the map.
- First posted on arXiv 31 Jan 2024.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Publicly Available Clinical BERT Embeddings](https://mlsanity.com/paper/1904.03323/) (2019)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings](https://mlsanity.com/paper/1909.00512/) (2019)
- [ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission](https://mlsanity.com/paper/1904.05342/) (2019)
- [TabLLM: Few-shot Classification of Tabular Data with Large Language Models](https://mlsanity.com/paper/2210.10723/) (2022)
- [BEHRT: Transformer for Electronic Health Records](https://mlsanity.com/paper/1907.09538/) (2019)
- [Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis](https://mlsanity.com/paper/1706.03446/) (2017)
- [EHRSHOT: An EHR Benchmark for Few-Shot Evaluation of Foundation Models](https://mlsanity.com/paper/2307.02028/) (2023)
- [Language Models Are An Effective Patient Representation Learning Technique For Electronic Health Record Data](https://mlsanity.com/paper/2001.05295/) (2020)
- [EHRAgent: Code Empowers Large Language Models for Few-shot Complex Tabular Reasoning on Electronic Health Records](https://mlsanity.com/paper/2401.07128/) (2024)
- [Hi-BEHRT: Hierarchical Transformer-based model for accurate prediction of clinical events using multimodal longitudinal electronic health records](https://mlsanity.com/paper/2106.11360/) (2021)
- [A Transformer-based representation-learning model with unified processing of multimodal input for clinical diagnostics](https://mlsanity.com/paper/2306.00864/) (2023)
- [GenHPF: General Healthcare Predictive Framework with Multi-task Multi-source Learning](https://mlsanity.com/paper/2207.09858/) (2022)
- [AMMU : A Survey of Transformer-based Biomedical Pretrained Language Models](https://mlsanity.com/paper/2105.00827/) (2021)
- [Benchmarking emergency department triage prediction models with machine learning and large public electronic health records](https://mlsanity.com/paper/2111.11017/) (2021)
- [TABLET: Learning From Instructions For Tabular Data](https://mlsanity.com/paper/2304.13188/) (2023)
- [Exploring Transformers in Natural Language Generation: GPT, BERT, and XLNet](https://mlsanity.com/paper/2102.08036/) (2021)
- [Unifying Heterogeneous Electronic Health Records Systems via Text-Based Code Embedding](https://mlsanity.com/paper/2108.03625/) (2021)
- [MediTab: Scaling Medical Tabular Data Predictors via Data Consolidation, Enrichment, and Refinement](https://mlsanity.com/paper/2305.12081/) (2023)
- [Multimodal Clinical Benchmark for Emergency Care (MC-BEC): A Comprehensive Benchmark for Evaluating Foundation Models in Emergency Medicine](https://mlsanity.com/paper/2311.04937/) (2023)
- [ExBEHRT: Extended Transformer for Electronic Health Records to Predict Disease Subtypes & Progressions](https://mlsanity.com/paper/2303.12364/) (2023)

## Built on by (most cited first, 22 of 22)

- [Clinical ModernBERT: An efficient and long context encoder for biomedical text](https://mlsanity.com/paper/2504.03964/) (2025)
- [Can Large Language Models abstract Medical Coded Language?](https://mlsanity.com/paper/2403.10822/) (2024)
- [A Case Study Exploring the Current Landscape of Synthetic Medical Record Generation with Commercial LLMs](https://mlsanity.com/paper/2504.14657/) (2025)
- [ChronoFormer: Time-Aware Transformer Architectures for Structured Clinical Event Modeling](https://mlsanity.com/paper/2504.07373/) (2025)
- [Structured Semantics from Unstructured Notes: Language Model Approaches to EHR-Based Decision Support](https://mlsanity.com/paper/2506.06340/) (2025)
- [FoMoH: A clinically meaningful foundation model evaluation for structured electronic health records](https://mlsanity.com/paper/2505.16941/) (2025)
- [Wavelet-Driven Masked Multiscale Reconstruction for PPG Foundation Models](https://mlsanity.com/paper/2601.12215/) (2026)
- [One Loss to Rule Them All: Marked Time-to-Event for Structured EHR Foundation Models](https://mlsanity.com/paper/2602.00541/) (2026)
- [Retrieval-aligned Tabular Foundation Models Enable Robust Clinical Risk Prediction in Electronic Health Records Under Real-world Constraints](https://mlsanity.com/paper/2604.01841/) (2026)
- [EHR2Path: Comprehensive Pathway-Level Modeling of Longitudinal Patient Trajectories from Multimodal Electronic Health Records](https://mlsanity.com/paper/2506.04831/) (2025)
- [Time-Aware Attention for Enhanced Electronic Health Records Modeling](https://mlsanity.com/paper/2507.14847/) (2025)
- [Learning Clinical Representations Under Systematic Distribution Shift](https://mlsanity.com/paper/2603.07348/) (2026)
- [Discriminative Representation Learning for Clinical Prediction](https://mlsanity.com/paper/2603.20921/) (2026)
- [Temporal Entailment Pretraining for Clinical Language Models over EHR Data](https://mlsanity.com/paper/2504.18128/) (2025)
- [The Multimodal Paradox: How Added and Missing Modalities Shape Bias and Performance in Multimodal AI](https://mlsanity.com/paper/2505.03020/) (2025)
- [KEEP: Integrating Medical Ontologies with Clinical Data for Robust Code Embeddings](https://mlsanity.com/paper/2510.05049/) (2025)
- [Learning Longitudinal Health Representations from EHR and Wearable Data](https://mlsanity.com/paper/2601.12227/) (2026)
- [Uncertainty-Aware Foundation Models for Clinical Data](https://mlsanity.com/paper/2604.04175/) (2026)
- [Predicting Post-Traumatic Epilepsy from Clinical Records using Large Language Model Embeddings](https://mlsanity.com/paper/2604.14547/) (2026)
- [Dense Feature Learning via Linear Structure Preservation in Medical Data](https://mlsanity.com/paper/2602.07706/) (2026)
- [Event Fields: Learning Latent Event Structure for Waveform Foundation Models](https://mlsanity.com/paper/2605.08685/) (2026)
- [WISTERIA: Learning Clinical Representations from Noisy Supervision via Multi-View Consistency in Electronic Health Records](https://mlsanity.com/paper/2605.09765/) (2026)

## Abstract (arXiv)

In this work, we introduce the Multiple Embedding Model for EHR (MEME), an approach that serializes multimodal EHR tabular data into text using pseudo-notes, mimicking clinical text generation. This conversion not only preserves better representations of categorical data and learns contexts but also enables the effective employment of pretrained foundation models for rich feature representation. To address potential issues with context length, our framework encodes embeddings for each EHR modality separately. We demonstrate the effectiveness of MEME by applying it to several decision support tasks within the Emergency Department across multiple hospital systems. Our findings indicate that MEME outperforms traditional machine learning, EHR-specific foundation models, and general LLMs, highlighting its potential as a general and extendible EHR representation strategy.