# The CRITICAL Records Integrated Standardization Pipeline (CRISP): End-to-End Processing of Large-scale Multi-institutional OMOP CDM Data

arXiv: https://arxiv.org/abs/2509.08247 · HTML: https://mlsanity.com/paper/2509.08247/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 33 references that are in the map.
- First posted on arXiv 10 Sep 2025.

## Builds on (most cited first, 10 of 10)

- [Recurrent Neural Networks for Multivariate Time Series with Missing Values](https://mlsanity.com/paper/1606.01865/) (2016)
- [Confounding variables can degrade generalization performance of radiological deep learning models](https://mlsanity.com/paper/1807.00431/) (2018)
- [Scalable and accurate deep learning for electronic health records](https://mlsanity.com/paper/1801.07860/) (2018)
- [MIMIC-Extract: A Data Extraction, Preprocessing, and Representation Pipeline for MIMIC-III](https://mlsanity.com/paper/1907.08322/) (2019)
- [An Extensive Data Processing Pipeline for MIMIC-IV](https://mlsanity.com/paper/2204.13841/) (2022)
- [Feature Robustness in Non-stationary Health Records: Caveats to Deployable Model Performance in Common Clinical Machine Learning Tasks](https://mlsanity.com/paper/1908.00690/) (2019)
- [Large Language Models are Powerful Electronic Health Record Encoders](https://mlsanity.com/paper/2502.17403/) (2025)
- [Temporal Pointwise Convolutional Networks for Length of Stay Prediction in the Intensive Care Unit](https://mlsanity.com/paper/2007.09483/) (2020)
- [Computing Extremely Accurate Quantiles Using t-Digests](https://mlsanity.com/paper/1902.04023/) (2019)
- [A Multidatabase ExTRaction PipEline (METRE) for Facile Cross Validation in Critical Care Research](https://mlsanity.com/paper/2302.13402/) (2023)

## Built on by (most cited first, 1 of 1)

- [Position: Medical AI Neglects Real Treatment Outcomes](https://mlsanity.com/paper/2608.14598/) (2026)

## Abstract (arXiv)

While existing critical care EHR datasets such as MIMIC and eICU have enabled significant advances in clinical AI research, the CRITICAL dataset opens new frontiers by providing extensive scale and diversity -- containing 1.95 billion records from 371,365 patients across four geographically diverse CTSA institutions. CRITICAL's unique strength lies in capturing full-spectrum patient journeys, including pre-ICU, ICU, and post-ICU encounters across both inpatient and outpatient settings. This multi-institutional, longitudinal perspective creates transformative opportunities for developing generalizable predictive models and advancing health equity research. However, the richness of this multi-site resource introduces substantial complexity in data harmonization, with heterogeneous collection practices and diverse vocabulary usage patterns requiring sophisticated preprocessing approaches. We present CRISP to unlock the full potential of this valuable resource. CRISP systematically transforms raw Observational Medical Outcomes Partnership Common Data Model data into ML-ready datasets through: (1) transparent data quality management with comprehensive audit trails, (2) cross-vocabulary mapping of heterogeneous medical terminologies to unified SNOMED-CT standards, with deduplication and unit standardization, (3) modular architecture with parallel optimization enabling complete dataset processing in <1 day even on standard computing hardware, and (4) comprehensive baseline model benchmarks spanning multiple clinical prediction tasks to establish reproducible performance standards. By providing processing pipeline, baseline implementations, and detailed transformation documentation, CRISP saves researchers months of preprocessing effort and democratizes access to large-scale multi-institutional critical care data, enabling them to focus on advancing clinical AI.