# Surrogate Substitution Preserves PHI Detectability: A Multi-Detector Equivalence Study

arXiv: https://arxiv.org/abs/2608.03172 · HTML: https://mlsanity.com/paper/2608.03172/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 47 references that are in the map.
- First posted on arXiv 4 Aug 2026.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model](https://mlsanity.com/paper/2405.04434/) (2024)
- [With Little Power Comes Great Responsibility](https://mlsanity.com/paper/2010.06595/) (2020)
- [DeID-GPT: Zero-shot Medical Text De-Identification by GPT-4](https://mlsanity.com/paper/2303.11032/) (2023)
- [Does BERT Pretrained on Clinical Notes Reveal Sensitive Data?](https://mlsanity.com/paper/2104.07762/) (2021)
- [De-identification of Patient Notes with Recurrent Neural Networks](https://mlsanity.com/paper/1606.03475/) (2016)
- [Large Language Models Are Not Strong Abstract Reasoners](https://mlsanity.com/paper/2305.19555/) (2023)
- [Deliberate Reasoning in Language Models as Structure-Aware Planning with an Accurate World Model](https://mlsanity.com/paper/2410.03136/) (2024)
- [MultiCoNER: A Large-scale Multilingual dataset for Complex Named Entity Recognition](https://mlsanity.com/paper/2208.14536/) (2022)
- [Privacy- and Utility-Preserving NLP with Anonymized Data: A case study of Pseudonymization](https://mlsanity.com/paper/2306.05561/) (2023)
- [De-identification is not enough: a comparison between de-identified and synthetic clinical notes](https://mlsanity.com/paper/2402.00179/) (2024)
- [Abstract Meaning Representation-Based Logic-Driven Data Augmentation for Logical Reasoning](https://mlsanity.com/paper/2305.12599/) (2023)
- [Enhancing Long Chain-of-Thought Reasoning through Multi-Path Plan Aggregation](https://mlsanity.com/paper/2510.11620/) (2025)
- [Adaptive Information Control for Search-Augmented LLM Reasoning](https://mlsanity.com/paper/2602.01672/) (2026)
- [Assessing and Enhancing the Robustness of Large Language Models with Task Structure Variations for Logical Reasoning](https://mlsanity.com/paper/2310.09430/) (2023)
- [MultiCoNER v2: a Large Multilingual dataset for Fine-grained and Noisy Named Entity Recognition](https://mlsanity.com/paper/2310.13213/) (2023)

## Abstract (arXiv)

Structure-preserving de-identification replaces each detected identifier with a realistic same-type surrogate ("Anna S." -> "Maria S.") rather than a tag such as [NAME]. If downstream PHI detectors no longer recognise surrogates, hiding-in-plain-sight protection and second-pass safety nets weaken. We ask: on the spans a de-identifier actually replaced, do detectors find the surrogate as often as the original value? We call this detectability preservation and treat it as a proxy, not a full measure of utility. Our paired protocol (i) scores detectability only on replaced spans, separately from coverage; (ii) runs an identical heterogeneous detector panel on original and transformed text; and (iii) bounds the paired change with document-level cluster-bootstrap equivalence testing and posterior probabilities instead of significance tests. For one commercial transform on 7 clinical and general-PII benchmarks in 7 languages (1,750 documents, 5,333 replaced spans, 11 detectors), masked-span recall moves from 75.8% to 74.6% (90% CI of the change [-1.62, -0.82] points; posterior probability > 0.99 of lying within +/-2 points). The change reaches 2-3 points on dense Spanish and Dutch text, and replaced spans are easier to detect than unreplaced ones (+16.6 points), so the result describes the replaced population only. Coverage is a separate, weaker axis (48% of gold PHI on MEDDOCAN). On financial, legal and enterprise text (750 documents, 10 detectors) the pooled result replicates (72.4% -> 72.8%), but rule-based and fine-tuned detectors lose 2-4 points on finance, and the transform edits substantial non-PII text. Redaction and Faker baselines give reference points. We release per-span predictions and analysis code.