# Can we generate portable representations for clinical time series data using LLMs?

arXiv: https://arxiv.org/abs/2603.23987 · HTML: https://mlsanity.com/paper/2603.23987/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 121 references that are in the map.
- First posted on arXiv 25 Mar 2026.

## Builds on (most cited first, 30 of 34)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Chronos: Learning the Language of Time Series](https://mlsanity.com/paper/2403.07815/) (2024)
- [Time-LLM: Time Series Forecasting by Reprogramming Large Language Models](https://mlsanity.com/paper/2310.01728/) (2023)
- [MTEB: Massive Text Embedding Benchmark](https://mlsanity.com/paper/2210.07316/) (2022)
- [One Fits All:Power General Time Series Analysis by Pretrained LM](https://mlsanity.com/paper/2302.11939/) (2023)
- [A Transformer-based Framework for Multivariate Time Series Representation Learning](https://mlsanity.com/paper/2010.02803/) (2020)
- [MedGemma Technical Report](https://mlsanity.com/paper/2507.05201/) (2025)
- [LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders](https://mlsanity.com/paper/2404.05961/) (2024)
- [TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting](https://mlsanity.com/paper/2405.14616/) (2024)
- [Time-Series Representation Learning via Temporal and Contextual Contrasting](https://mlsanity.com/paper/2106.14112/) (2021)
- [One Embedder, Any Task: Instruction-Finetuned Text Embeddings](https://mlsanity.com/paper/2212.09741/) (2022)
- [Improving Text Embeddings with Large Language Models](https://mlsanity.com/paper/2401.00368/) (2023)
- [TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting](https://mlsanity.com/paper/2310.04948/) (2023)
- [TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Series](https://mlsanity.com/paper/2308.08241/) (2023)
- [Large Language Models are Few-Shot Health Learners](https://mlsanity.com/paper/2305.15525/) (2023)
- [CALF: Aligning LLMs for Time Series Forecasting via Cross-modal Fine-Tuning](https://mlsanity.com/paper/2403.07300/) (2024)
- [Gecko: Versatile Text Embeddings Distilled from Large Language Models](https://mlsanity.com/paper/2403.20327/) (2024)
- [Time Series Prediction using Deep Learning Methods in Healthcare](https://mlsanity.com/paper/2108.13461/) (2021)
- [AutoTimes: Autoregressive Time Series Forecasters via Large Language Models](https://mlsanity.com/paper/2402.02370/) (2024)
- [HiRID-ICU-Benchmark -- A Comprehensive Machine Learning Benchmark on High-resolution ICU Data](https://mlsanity.com/paper/2111.08536/) (2021)
- [TOTEM: TOkenized Time Series EMbeddings for General Time Series Analysis](https://mlsanity.com/paper/2402.16412/) (2024)
- [Universal Time-Series Representation Learning: A Survey](https://mlsanity.com/paper/2401.03717/) (2024)
- [Yet Another ICU Benchmark: A Flexible Multi-Center Framework for Clinical ML](https://mlsanity.com/paper/2306.05109/) (2023)
- [When Raw Data Prevails: Are Large Language Model Embeddings Effective in Numerical Data Representation for Medical Machine Learning Applications?](https://mlsanity.com/paper/2408.11854/) (2024)
- [DAIQ: Auditing Demographic Attribute Inference from Question in LLMs](https://mlsanity.com/paper/2508.15830/) (2025)
- [Distributionally Robust Receive Combining](https://mlsanity.com/paper/2401.12345/) (2024)
- [Towards Foundation Models for Critical Care Time Series](https://mlsanity.com/paper/2411.16346/) (2024)

## Built on by (most cited first, 4 of 4)

- [Beyond the Training Distribution: Evaluating Predictions Under Distribution Shift and Selection Bias](https://mlsanity.com/paper/2606.14506/) (2026)
- [Learning to Select Source-Traceable Evidence for Language-Model Prediction from Irregular Clinical Time Series](https://mlsanity.com/paper/2605.20292/) (2026)
- [TRIAGE: Dialectical Reasoning for Explainable Risk Prediction on Irregularly Sampled Medical Time Series with LLMs](https://mlsanity.com/paper/2606.09030/) (2026)
- [LineupRL: Verifiable Reinforcement Learning for Time Series Captioning via Caption-to-Series Identification](https://mlsanity.com/paper/2610.01800/) (2026)

## Abstract (arXiv)

Deploying clinical ML is slow and brittle: models that work at one hospital often degrade under distribution shifts at the next. In this work, we study a simple question -- can large language models (LLMs) create portable patient embeddings i.e. representations of patients enable a downstream predictor built on one hospital to be used elsewhere with minimal-to-no retraining and fine-tuning. To do so, we map from irregular ICU time series onto concise natural language summaries using a frozen LLM, then embed each summary with a frozen text embedding model to obtain a fixed length vector capable of serving as input to a variety of downstream predictors. Across three cohorts (MIMIC-IV, HIRID, PPICU), on multiple clinically grounded forecasting and classification tasks, we find that our approach is simple, easy to use and competitive with in-distribution with grid imputation, self-supervised representation learning, and time series foundation models, while exhibiting smaller relative performance drops when transferring to new hospitals. We study the variation in performance across prompt design, with structured prompts being crucial to reducing the variance of the predictive models without altering mean accuracy. We find that using these portable representations improves few-shot learning and does not increase demographic recoverability of age or sex relative to baselines, suggesting little additional privacy risk. Our work points to the potential that LLMs hold as tools to enable the scalable deployment of production grade predictive models by reducing the engineering overhead.