Surrogate Substitution Preserves PHI Detectability: A Multi-Detector Equivalence Study
Not on the map. Why?
Abstract (arXiv)
Structure-preserving de-identification replaces each detected identifier with a realistic same-type surrogate ("Anna S." -> "Maria S.") rather than a tag such as [NAME]. If downstream PHI detectors no longer recognise surrogates, hiding-in-plain-sight protection and second-pass safety nets weaken. We ask: on the spans a de-identifier actually replaced, do detectors find the surrogate as often as the original value? We call this detectability preservation and treat it as a proxy, not a full measure of utility. Our paired protocol (i) scores detectability only on replaced spans, separately from coverage; (ii) runs an identical heterogeneous detector panel on original and transformed text; and (iii) bounds the paired change with document-level cluster-bootstrap equivalence testing and posterior probabilities instead of significance tests. For one commercial transform on 7 clinical and general-PII benchmarks in 7 languages (1,750 documents, 5,333 replaced spans, 11 detectors), masked-span recall moves from 75.8% to 74.6% (90% CI of the change [-1.62, -0.82] points; posterior probability > 0.99 of lying within +/-2 points). The change reaches 2-3 points on dense Spanish and Dutch text, and replaced spans are easier to detect than unreplaced ones (+16.6 points), so the result describes the replaced population only. Coverage is a separate, weaker axis (48% of gold PHI on MEDDOCAN). On financial, legal and enterprise text (750 documents, 10 detectors) the pooled result replicates (72.4% -> 72.8%), but rule-based and fine-tuned detectors lose 2-4 points on finance, and the transform edits substantial non-PII text. Redaction and Faker baselines give reference points. We release per-span predictions and analysis code.
Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 23 of its 47 references that are in the map. First posted on arXiv 4 Aug 2026.
Builds on
23 of 47 references.
References · 23
- 2017Attention Is All You Needcited by 60,106
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
- 2023GPT-4 Technical Reportcited by 18,278
- 2024The Llama 3 Herd of Modelscited by 14,486
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,322
- 2024Qwen2.5 Technical Reportcited by 4,840
- 2020Extracting Training Data from Large Language Modelscited by 2,181
- 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,968
- 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,115
- 2020With Little Power Comes Great Responsibilitycited by 140
- 2023DeID-GPT: Zero-shot Medical Text De-Identification by GPT-4cited by 129
- 2021Does BERT Pretrained on Clinical Notes Reveal Sensitive Data?cited by 93
11 more
- 2016De-identification of Patient Notes with Recurrent Neural Networkscited by 88
- 2023Large Language Models Are Not Strong Abstract Reasonerscited by 53
- 2024Deliberate Reasoning in Language Models as Structure-Aware Planning with an Accurate World Modelcited by 22
- 2022MultiCoNER: A Large-scale Multilingual dataset for Complex Named Entity Recognitioncited by 19
- 2023Privacy- and Utility-Preserving NLP with Anonymized Data: A case study of Pseudonymizationcited by 18
- 2024De-identification is not enough: a comparison between de-identified and synthetic clinical notescited by 14
- 2023Abstract Meaning Representation-Based Logic-Driven Data Augmentation for Logical Reasoningcited by 11
- 2025Enhancing Long Chain-of-Thought Reasoning through Multi-Path Plan Aggregationcited by 11
- 2023Assessing and Enhancing the Robustness of Large Language Models with Task Structure Variations for Logical Reasoningcited by 10
- 2026Adaptive Information Control for Search-Augmented LLM Reasoningcited by 9
- 2023MultiCoNER v2: a Large Multilingual dataset for Fine-grained and Noisy Named Entity Recognitioncited by 2
Built on by
Not cited yet.