# FineEHR: Refine Clinical Note Representations to Improve Mortality Prediction

arXiv: https://arxiv.org/abs/2304.11794 · HTML: https://mlsanity.com/paper/2304.11794/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 44 references that are in the map.
- First posted on arXiv 24 Apr 2023.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Align before Fuse: Vision and Language Representation Learning with Momentum Distillation](https://mlsanity.com/paper/2107.07651/) (2021)
- [Fully-Convolutional Siamese Networks for Object Tracking](https://mlsanity.com/paper/1606.09549/) (2016)
- [Recurrent Neural Networks for Multivariate Time Series with Missing Values](https://mlsanity.com/paper/1606.01865/) (2016)
- [ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission](https://mlsanity.com/paper/1904.05342/) (2019)
- [Multitask learning and benchmarking with clinical time series data](https://mlsanity.com/paper/1703.07771/) (2017)
- [Multi-Time Attention Networks for Irregularly Sampled Time Series](https://mlsanity.com/paper/2101.10318/) (2021)
- [A Convolutional Neural Network for Modelling Sentences](https://mlsanity.com/paper/1404.2188/) (2014)
- [Medical Image Analysis using Convolutional Neural Networks: A Review](https://mlsanity.com/paper/1709.02250/) (2017)
- [Using Clinical Notes with Time Series Data for ICU Management](https://mlsanity.com/paper/1909.09702/) (2019)
- [Heterogeneous Similarity Graph Neural Network on Electronic Health Records](https://mlsanity.com/paper/2101.06800/) (2021)
- [GL-RG: Global-Local Representation Granularity for Video Captioning](https://mlsanity.com/paper/2205.10706/) (2022)
- [Improving Hospital Mortality Prediction with Medical Named Entities and Multimodal Learning](https://mlsanity.com/paper/1811.12276/) (2018)
- [TopoTxR: A Topological Biomarker for Predicting Treatment Response in Breast Cancer](https://mlsanity.com/paper/2105.06049/) (2021)
- [BotTriNet: A Unified and Efficient Embedding for Social Bots Detection via Metric Learning](https://mlsanity.com/paper/2304.03144/) (2023)
- [BotShape: A Novel Social Bots Detection Approach via Behavioral Patterns](https://mlsanity.com/paper/2303.10214/) (2023)
- [DeepMTL Pro: Deep Learning Based MultipleTransmitter Localization and Power Estimation](https://mlsanity.com/paper/2112.13181/) (2021)
- [Mitigating Health Disparities in EHR via Deconfounder](https://mlsanity.com/paper/2210.15901/) (2022)
- [Robust Data Preprocessing for Machine-Learning-Based Disk Failure Prediction in Cloud Production Environments](https://mlsanity.com/paper/1912.09722/) (2019)
- [Detecting and measuring human gastric peristalsis using magnetically controlled capsule endoscope](https://mlsanity.com/paper/2301.10218/) (2023)

## Built on by (most cited first, 9 of 9)

- [Task-Agnostic Detector for Insertion-Based Backdoor Attacks](https://mlsanity.com/paper/2403.17155/) (2024)
- [CALICO: Self-Supervised Camera-LiDAR Contrastive Pre-training for BEV Perception](https://mlsanity.com/paper/2306.00349/) (2023)
- [MedLens: Improve Mortality Prediction Via Medical Signs Selecting and Regression](https://mlsanity.com/paper/2305.11742/) (2023)
- [An Integrative Paradigm for Enhanced Stroke Prediction: Synergizing XGBoost and xDeepFM Algorithms](https://mlsanity.com/paper/2310.16430/) (2023)
- [FakeSwarm: Improving Fake News Detection with Swarming Characteristics](https://mlsanity.com/paper/2305.19194/) (2023)
- [A Counterfactual Fair Model for Longitudinal Electronic Health Records via Deconfounder](https://mlsanity.com/paper/2308.11819/) (2023)
- [SafeAug: Safety-Critical Driving Data Augmentation from Naturalistic Datasets](https://mlsanity.com/paper/2501.02143/) (2025)
- [ClinStructor: AI-Powered Structuring of Unstructured Clinical Texts](https://mlsanity.com/paper/2511.11883/) (2025)
- [Word Definitions from Large Language Models](https://mlsanity.com/paper/2311.06362/) (2023)

## Abstract (arXiv)

Monitoring the health status of patients in the Intensive Care Unit (ICU) is a critical aspect of providing superior care and treatment. The availability of large-scale electronic health records (EHR) provides machine learning models with an abundance of clinical text and vital sign data, enabling them to make highly accurate predictions. Despite the emergence of advanced Natural Language Processing (NLP) algorithms for clinical note analysis, the complex textual structure and noise present in raw clinical data have posed significant challenges. Coarse embedding approaches without domain-specific refinement have limited the accuracy of these algorithms. To address this issue, we propose FINEEHR, a system that utilizes two representation learning techniques, namely metric learning and fine-tuning, to refine clinical note embeddings, while leveraging the intrinsic correlations among different health statuses and note categories. We evaluate the performance of FINEEHR using two metrics, namely Area Under the Curve (AUC) and AUC-PR, on a real-world MIMIC III dataset. Our experimental results demonstrate that both refinement approaches improve prediction accuracy, and their combination yields the best results. Moreover, our proposed method outperforms prior works, with an AUC improvement of over 10%, achieving an average AUC of 96.04% and an average AUC-PR of 96.48% across various classifiers.