# Re-identification of De-identified Documents with Autoregressive Infilling

arXiv: https://arxiv.org/abs/2505.12859 · HTML: https://mlsanity.com/paper/2505.12859/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 44 references that are in the map.
- First posted on arXiv 19 May 2025.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Improving language models by retrieving from trillions of tokens](https://mlsanity.com/paper/2112.04426/) (2021)
- [ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT](https://mlsanity.com/paper/2004.12832/) (2020)
- [GLM: General Language Model Pretraining with Autoregressive Blank Infilling](https://mlsanity.com/paper/2103.10360/) (2021)
- [Atlas: Few-shot Learning with Retrieval Augmented Language Models](https://mlsanity.com/paper/2208.03299/) (2022)
- [In-Context Retrieval-Augmented Language Models](https://mlsanity.com/paper/2302.00083/) (2023)
- [Efficient Training of Language Models to Fill in the Middle](https://mlsanity.com/paper/2207.14255/) (2022)
- [DeID-GPT: Zero-shot Medical Text De-Identification by GPT-4](https://mlsanity.com/paper/2303.11032/) (2023)
- [Enabling Language Models to Fill in the Blanks](https://mlsanity.com/paper/2005.05339/) (2020)
- [The Text Anonymization Benchmark (TAB): A Dedicated Corpus and Evaluation Framework for Text Anonymization](https://mlsanity.com/paper/2202.00443/) (2022)
- [De-identification of Patient Notes with Recurrent Neural Networks](https://mlsanity.com/paper/1606.03475/) (2016)
- [C-sanitized: a privacy model for document redaction and sanitization](https://mlsanity.com/paper/1406.4285/) (2014)
- [No Intruder, no Validity: Evaluation Criteria for Privacy-Preserving Text Anonymization](https://mlsanity.com/paper/2103.09263/) (2021)
- [BMX: Entropy-weighted Similarity and Semantic-enhanced Lexical Search](https://mlsanity.com/paper/2408.06643/) (2024)
- [DIRI: Adversarial Patient Reidentification with Large Language Models for Evaluating Clinical Text Anonymization](https://mlsanity.com/paper/2410.17035/) (2024)

## Built on by (most cited first, 3 of 3)

- [Stronger Re-identification Attacks through Reasoning and Aggregation](https://mlsanity.com/paper/2510.09184/) (2025)
- [The Conundrum of Trustworthy Research on Attacking Personally Identifiable Information Removal Techniques](https://mlsanity.com/paper/2603.08207/) (2026)
- [Aggregation-Aware Synthetic Text Generation Against Authorship Re-Identification](https://mlsanity.com/paper/2608.22161/) (2026)

## Abstract (arXiv)

Documents revealing sensitive information about individuals must typically be de-identified. This de-identification is often done by masking all mentions of personally identifiable information (PII), thereby making it more difficult to uncover the identity of the person(s) in question. To investigate the robustness of de-identification methods, we present a novel, RAG-inspired approach that attempts the reverse process of re-identification based on a database of documents representing background knowledge. Given a text in which personal identifiers have been masked, the re-identification proceeds in two steps. A retriever first selects from the background knowledge passages deemed relevant for the re-identification. Those passages are then provided to an infilling model which seeks to infer the original content of each text span. This process is repeated until all masked spans are replaced. We evaluate the re-identification on three datasets (Wikipedia biographies, court rulings and clinical notes). Results show that (1) as many as 80% of de-identified text spans can be successfully recovered and (2) the re-identification accuracy increases along with the level of background knowledge.