# DIRI: Adversarial Patient Reidentification with Large Language Models for Evaluating Clinical Text Anonymization

arXiv: https://arxiv.org/abs/2410.17035 · HTML: https://mlsanity.com/paper/2410.17035/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 29 references that are in the map.
- First posted on arXiv 22 Oct 2024.

## Builds on (most cited first, 9 of 9)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission](https://mlsanity.com/paper/1904.05342/) (2019)
- [TAPAS: Weakly Supervised Table Parsing via Pre-training](https://mlsanity.com/paper/2004.02349/) (2020)
- [The Text Anonymization Benchmark (TAB): A Dedicated Corpus and Evaluation Framework for Text Anonymization](https://mlsanity.com/paper/2202.00443/) (2022)
- [A new paradigm for accelerating clinical data science at Stanford Medicine](https://mlsanity.com/paper/2003.10534/) (2020)
- [Unsupervised Text Deidentification](https://mlsanity.com/paper/2210.11528/) (2022)
- [Comparing Rule-based, Feature-based and Deep Neural Methods for De-identification of Dutch Medical Records](https://mlsanity.com/paper/2001.05714/) (2020)

## Built on by (most cited first, 4 of 4)

- [A False Sense of Privacy: Evaluating Textual Data Sanitization Beyond Surface-level Privacy Leakage](https://mlsanity.com/paper/2504.21035/) (2025)
- [Re-identification of De-identified Documents with Autoregressive Infilling](https://mlsanity.com/paper/2505.12859/) (2025)
- [VSF-Med:A Vulnerability Scoring Framework for Medical Vision-Language Models](https://mlsanity.com/paper/2507.00052/) (2025)
- [Context-Aware Classification and Grading of Sensitive Information in Online Conversational Health Data](https://mlsanity.com/paper/2601.09717/) (2025)

## Abstract (arXiv)

Sharing protected health information (PHI) is critical for furthering biomedical research. Before data can be distributed, practitioners often perform deidentification to remove any PHI contained in the text. Contemporary deidentification methods are evaluated on highly saturated datasets (tools achieve near-perfect accuracy) which may not reflect the full variability or complexity of real-world clinical text and annotating them is resource intensive, which is a barrier to real-world applications. To address this gap, we developed an adversarial approach using a large language model (LLM) to re-identify the patient corresponding to a redacted clinical note and evaluated the performance with a novel De-Identification/Re-Identification (DIRI) method. Our method uses a large language model to reidentify the patient corresponding to a redacted clinical note. We demonstrate our method on medical data from Weill Cornell Medicine anonymized with three deidentification tools: rule-based Philter and two deep-learning-based models, BiLSTM-CRF and ClinicalBERT. Although ClinicalBERT was the most effective, masking all identified PII, our tool still reidentified 9% of clinical notes Our study highlights significant weaknesses in current deidentification technologies while providing a tool for iterative development and improvement.