# No Intruder, no Validity: Evaluation Criteria for Privacy-Preserving Text Anonymization

arXiv: https://arxiv.org/abs/2103.09263 · HTML: https://mlsanity.com/paper/2103.09263/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 12 references that are in the map.
- First posted on arXiv 16 Mar 2021.

## Builds on (most cited first, 1 of 1)

- [Privacy- and Utility-Preserving Textual Analysis via Calibrated Multivariate Perturbations](https://mlsanity.com/paper/1910.08902/) (2019)

## Built on by (most cited first, 13 of 13)

- [The Text Anonymization Benchmark (TAB): A Dedicated Corpus and Evaluation Framework for Text Anonymization](https://mlsanity.com/paper/2202.00443/) (2022)
- [Just What do You Think You're Doing, Dave?' A Checklist for Responsible Data Use in NLP](https://mlsanity.com/paper/2109.06598/) (2021)
- [Data Governance in the Age of Large-Scale Data-Driven Language Technology](https://mlsanity.com/paper/2206.03216/) (2022)
- [Textwash -- automated open-source text anonymisation](https://mlsanity.com/paper/2208.13081/) (2022)
- [Robust Utility-Preserving Text Anonymization Based on Large Language Models](https://mlsanity.com/paper/2407.11770/) (2024)
- [Neural Text Sanitization with Privacy Risk Indicators: An Empirical Analysis](https://mlsanity.com/paper/2310.14312/) (2023)
- [A Survey on Current Trends and Recent Advances in Text Anonymization](https://mlsanity.com/paper/2508.21587/) (2025)
- [How do we measure privacy in text? A survey of text anonymization metrics](https://mlsanity.com/paper/2512.01109/) (2025)
- [RAI4IoE: Responsible AI for Enabling the Internet of Energy](https://mlsanity.com/paper/2309.11691/) (2023)
- [Re-identification of De-identified Documents with Autoregressive Infilling](https://mlsanity.com/paper/2505.12859/) (2025)
- [Secret-Keeping in Question Answering](https://mlsanity.com/paper/2303.09067/) (2023)
- [Stronger Re-identification Attacks through Reasoning and Aggregation](https://mlsanity.com/paper/2510.09184/) (2025)
- [The Text Beside the Image: Detection, Utility and Leakage for Trustworthy Multimodal Medical Data and Beyond](https://mlsanity.com/paper/2609.33280/) (2026)

## Abstract (arXiv)

For sensitive text data to be shared among NLP researchers and practitioners, shared documents need to comply with data protection and privacy laws. There is hence a growing interest in automated approaches for text anonymization. However, measuring such methods' performance is challenging: missing a single identifying attribute can reveal an individual's identity. In this paper, we draw attention to this problem and argue that researchers and practitioners developing automated text anonymization systems should carefully assess whether their evaluation methods truly reflect the system's ability to protect individuals from being re-identified. We then propose TILD, a set of evaluation criteria that comprises an anonymization method's technical performance, the information loss resulting from its anonymization, and the human ability to de-anonymize redacted documents. These criteria may facilitate progress towards a standardized way for measuring anonymization performance.