mlsanityData to 7 Oct 2026

Quantifying surprise in clinical care: Detecting highly informative events in electronic health records with foundation models

arXiv 2507.22798 ↗First posted 30 Jul 2025cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

We present a foundation model-derived method to identify highly informative tokens and events in electronic health records. Our approach considers incoming data in the entire context of a patient's hospitalization and so can flag anomalous events that rule-based approaches would consider within a normal range. We demonstrate that the events our model flags are significant for predicting downstream patient outcomes and that a fraction of events identified as carrying little information can safely be dropped. Additionally, we show how informativeness can help interpret the predictions of prognostic models trained on foundation model-derived representations.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 25 of its 65 references that are in the map. First posted on arXiv 30 Jul 2025.

Builds on

25 of 65 references.

References · 25

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  5. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,455
  6. 2023Model Tells You What to Discard: Adaptive KV Cache Compression for LLMscited by 466
  7. 2023Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Timecited by 428
  8. 2023Hyena Hierarchy: Towards Larger Convolutional Language Modelscited by 388
  9. 2020Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease predictioncited by 368
  10. 2018Scalable and accurate deep learning for electronic health recordscited by 275
  11. 2019BEHRT: Transformer for Electronic Health Recordscited by 258
  12. 2015Doctor AI: Predicting Clinical Events via Recurrent Neural Networkscited by 205
13 more
  1. 2023EHRSHOT: An EHR Benchmark for Few-Shot Evaluation of Foundation Modelscited by 101
  2. 2015Learning to Diagnose with LSTM Recurrent Neural Networkscited by 99
  3. 2020Language Models Are An Effective Patient Representation Learning Technique For Electronic Health Record Datacited by 85
  4. 2021CEHR-BERT: Incorporating temporal information from structured EHR data to improve prediction taskscited by 75
  5. 2023MOTOR: A Time-To-Event Foundation Model For Structured Medical Recordscited by 75
  6. 2024Zero Shot Health Trajectory Prediction Using Transformercited by 74
  7. 2024Attention Score is not All You Need for Token Importance Indicator in KV Cache Reduction: Value Also Matterscited by 73
  8. 2023Event Stream GPT: A Data Pre-processing and Modeling Library for Generative, Pre-trained Transformers over Continuous-time Sequences of Complex Eventscited by 46
  9. 2024Context Clues: Evaluating Long Context Models for Clinical Prediction Tasks on EHRscited by 39
  10. 2024EHRMamba: Towards Generalizable and Scalable Foundation Models for Electronic Health Recordscited by 31
  11. 2024ClinicalMamba: A Generative Clinical Language Model on Longitudinal Clinical Notescited by 22
  12. 2025Foundation Model of Electronic Medical Records for Adaptive Risk Estimationcited by 17
  13. 2025Foundation models for electronic health records: representation dynamics and transferabilitycited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Efficient Generative Prediction for EHR Foundation Models: The SCOPE and REACH Estimatorscited by 2
  2. 2026Representation Before Training: A Practical Benchmark for Generative Medical Event Model Tokenizationcited by 0
  3. 2026Federated generative event models for tokenized electronic health recordscited by 0