mlsanityData to 14 Oct 2026

Revisiting the MIMIC-IV Benchmark: Experiments Using Language Models for Electronic Health Records

arXiv 2504.20547 ↗First posted 29 Apr 2025cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

The lack of standardized evaluation benchmarks in the medical domain for text inputs can be a barrier to widely adopting and leveraging the potential of natural language models for health-related downstream tasks. This paper revisited an openly available MIMIC-IV benchmark for electronic health records (EHRs) to address this issue. First, we integrate the MIMIC-IV data within the Hugging Face datasets library to allow an easy share and use of this collection. Second, we investigate the application of templates to convert EHR tabular data to text. Experiments using fine-tuned and zero-shot LLMs on the mortality of patients task show that fine-tuned text-based models are competitive against robust tabular classifiers. In contrast, zero-shot LLMs struggle to leverage EHR representations. This study underlines the potential of text-based approaches in the medical field and highlights areas for further improvement.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 21 of its 32 references that are in the map. First posted on arXiv 29 Apr 2025.

Builds on

21 of 32 references.

References · 21

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  6. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  7. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  8. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
  9. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 1,009
  10. 2019Publicly Available Clinical BERT Embeddingscited by 1,002
  11. 2022A Survey on In-context Learningcited by 721
  12. 2019Release Strategies and the Social Impacts of Language Modelscited by 513
9 more
  1. 2020Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease predictioncited by 373
  2. 2017Multitask learning and benchmarking with clinical time series datacited by 290
  3. 2018Lessons from Natural Language Inference in the Clinical Domaincited by 151
  4. 2019MIMIC-Extract: A Data Extraction, Preprocessing, and Representation Pipeline for MIMIC-IIIcited by 122
  5. 2017Attend and Diagnose: Clinical Time Series Analysis using Attention Modelscited by 105
  6. 2020Text-to-Text Pre-Training for Data-to-Text Taskscited by 104
  7. 2019Learning the Graphical Structure of Electronic Health Records with Graph Convolutional Transformercited by 88
  8. 2022An Extensive Data Processing Pipeline for MIMIC-IVcited by 40
  9. 2021Simplified Data Wrangling with ir_datasetscited by 27

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.

  1. 2026EHR-RAG: Bridging Long-Horizon Structured Electronic Health Records and Large Language Models via Enhanced Retrieval-Augmented Generationcited by 6
  2. 2025EHR2Path: Comprehensive Pathway-Level Modeling of Longitudinal Patient Trajectories from Multimodal Electronic Health Recordscited by 3
  3. 2025SurvBench: A Standardised Preprocessing Pipeline for Multi-Modal Electronic Health Record Survival Analysiscited by 2
  4. 2025Stable Prediction of Adverse Events in Medical Time-Series Datacited by 0
  5. 2026DT-ICU: Towards Explainable Digital Twins for ICU Patient Monitoring via Multi-Modal and Multi-Task Iterative Inferencecited by 0