# PORTER: Language-Grounded Event Representations for Portable Structured EHR Foundation Models

arXiv: https://arxiv.org/abs/2606.24102 · HTML: https://mlsanity.com/paper/2606.24102/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 52 references that are in the map.
- First posted on arXiv 23 Jun 2026.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [Generating Long Sequences with Sparse Transformers](https://mlsanity.com/paper/1904.10509/) (2019)
- [Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models](https://mlsanity.com/paper/2506.05176/) (2025)
- [M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation](https://mlsanity.com/paper/2402.03216/) (2024)
- [MTEB: Massive Text Embedding Benchmark](https://mlsanity.com/paper/2210.07316/) (2022)
- [Scaling Data-Constrained Language Models](https://mlsanity.com/paper/2305.16264/) (2023)
- [Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease prediction](https://mlsanity.com/paper/2005.12833/) (2020)
- [Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment](https://mlsanity.com/paper/2312.12148/) (2023)
- [Language Models Are An Effective Patient Representation Learning Technique For Electronic Health Record Data](https://mlsanity.com/paper/2001.05295/) (2020)
- [MOTOR: A Time-To-Event Foundation Model For Structured Medical Records](https://mlsanity.com/paper/2301.03150/) (2023)
- [Zero Shot Health Trajectory Prediction Using Transformer](https://mlsanity.com/paper/2407.21124/) (2024)
- [GenHPF: General Healthcare Predictive Framework with Multi-task Multi-source Learning](https://mlsanity.com/paper/2207.09858/) (2022)
- [TIMER: Temporal Instruction Modeling and Evaluation for Longitudinal Clinical Records](https://mlsanity.com/paper/2503.04176/) (2025)
- [A Multi-Center Study on the Adaptability of a Shared Foundation Model for Electronic Health Records](https://mlsanity.com/paper/2311.11483/) (2023)
- [Large Language Models are Powerful Electronic Health Record Encoders](https://mlsanity.com/paper/2502.17403/) (2025)
- [Unifying Heterogeneous Electronic Health Records Systems via Text-Based Code Embedding](https://mlsanity.com/paper/2108.03625/) (2021)
- [MoMA: A Mixture-of-Multimodal-Agents Architecture for Enhancing Clinical Prediction Modelling](https://mlsanity.com/paper/2508.05492/) (2025)
- [Tokenization Tradeoffs in Structured EHR Foundation Models](https://mlsanity.com/paper/2603.15644/) (2026)
- [Representation learning to advance multi-institutional studies with electronic health record data from US and France](https://mlsanity.com/paper/2502.08547/) (2025)
- [One Loss to Rule Them All: Marked Time-to-Event for Structured EHR Foundation Models](https://mlsanity.com/paper/2602.00541/) (2026)
- [meds_reader: A fast and efficient EHR processing library](https://mlsanity.com/paper/2409.09095/) (2024)
- [CEHR-XGPT: A Scalable Multi-Task Foundation Model for Electronic Health Records](https://mlsanity.com/paper/2509.03643/) (2025)
- [Rethinking Tokenization for Clinical Time Series: When Less is More](https://mlsanity.com/paper/2512.05217/) (2025)
- [LLMs can construct powerful representations and streamline sample-efficient supervised learning](https://mlsanity.com/paper/2603.11679/) (2026)
- [Multi-lingual Multi-institutional Electronic Health Record based Predictive Model](https://mlsanity.com/paper/2604.00027/) (2026)

## Built on by (most cited first, 2 of 2)

- [Federated generative event models for tokenized electronic health records](https://mlsanity.com/paper/2608.02939/) (2026)
- [EHRAdapt: Adapting Pretrained Language Models to Electronic Health Records with Semantic Priors for Rare Clinical Events](https://mlsanity.com/paper/2609.34007/) (2026)

## Abstract (arXiv)

Most electronic health record (EHR) foundation models encode clinical events as discrete event tokens from a fixed vocabulary and therefore cannot directly represent events containing unseen concepts or new combinations of concepts and attributes such as numeric values. This limits transfer across institutions and even across deployment pipelines within the same institution. We introduce PORTER, a language-grounded structured EHR foundation model that decouples event representation from this fixed vocabulary. PORTER represents events through their descriptions using a frozen text encoder, integrates numeric values through a dedicated pathway, and learns clinical dynamics over patient timelines with an autoregressively pretrained temporal backbone. Across 74 clinical prediction tasks at a pediatric hospital, PORTER matched the mean AUROC of a fixed-vocabulary model with the same temporal backbone and pretraining objective. When the same patient timelines were rendered using event descriptions not seen during pretraining, PORTER transferred without retraining or vocabulary mapping, recovering 97.1% of the mean AUROC of a model trained directly on the target vocabulary. When transferred to MIMIC, PORTER outperformed the fixed-vocabulary model, which dropped 69% of events because their tokens were unseen. Mechanistic analyses showed cross-vocabulary transfer tracked preservation of patient-level representation geometry rather than the scale of the text encoder, and the numeric pathway improved sensitivity to magnitude without disrupting clinical concept identity. PORTER also achieved higher AUROC than a task-specific text serialization comparator, at 329-fold lower amortized compute. PORTER is a step toward vocabulary-independent EHR foundation models that reduce the need for vocabulary harmonization while preserving in-domain performance and enabling efficient cross-task reuse.