mlsanityData to 7 Oct 2026

The Text Anonymization Benchmark (TAB): A Dedicated Corpus and Evaluation Framework for Text Anonymization

arXiv 2202.00443 ↗First posted 25 Jan 2022cs.CL cs.AIcited by 94

Not on the map. Why?

Abstract (arXiv)

We present a novel benchmark and associated evaluation metrics for assessing the performance of text anonymization methods. Text anonymization, defined as the task of editing a text document to prevent the disclosure of personal information, currently suffers from a shortage of privacy-oriented annotated text resources, making it difficult to properly evaluate the level of privacy protection offered by various anonymization methods. This paper presents TAB (Text Anonymization Benchmark), a new, open-source annotated corpus developed to address this shortage. The corpus comprises 1,268 English-language court cases from the European Court of Human Rights (ECHR) enriched with comprehensive annotations about the personal information appearing in each document, including their semantic category, identifier type, confidential attributes, and co-reference relations. Compared to previous work, the TAB corpus is designed to go beyond traditional de-identification (which is limited to the detection of predefined semantic categories), and explicitly marks which text spans ought to be masked in order to conceal the identity of the person to be protected. Along with presenting the corpus and its annotation layers, we also propose a set of evaluation metrics that are specifically tailored towards measuring the performance of text anonymization, both in terms of privacy protection and utility preservation. We illustrate the use of the benchmark and the proposed metrics by assessing the empirical performance of several baseline text anonymization models. The full corpus along with its privacy-oriented annotation guidelines, evaluation scripts and baseline models are available on: https://github.com/NorskRegnesentral/text-anonymisation-benchmark

Key facts. Cited by 94 papers in the map (data through 7 Oct 2026). Builds on 26 of its 116 references that are in the map. First posted on arXiv 25 Jan 2022.

Builds on

26 of 116 references.

References · 26

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2020Longformer: The Long-Document Transformercited by 3,362
  4. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  5. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  6. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,404
  7. 2016Neural Architectures for Named Entity Recognitioncited by 548
  8. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 405
  9. 2017End-to-end Neural Coreference Resolutioncited by 240
  10. 2015Named Entity Recognition with Bidirectional LSTM-CNNscited by 201
  11. 2016Demographic Dialectal Variation in Social Media: A Case Study of African-American Englishcited by 194
  12. 2018Adversarial Removal of Demographic Attributes from Text Datacited by 166
14 more
  1. 2016De-identification of Patient Notes with Recurrent Neural Networkscited by 88
  2. 2018Towards Robust and Privacy-preserving Text Representationscited by 75
  3. 2018Generalised Differential Privacy for Text Document Processingcited by 52
  4. 2019Leveraging Hierarchical Representations for Preserving Privacy and Utility in Textcited by 49
  5. 2020TextHide: Tackling Data Privacy in Language Understanding Taskscited by 32
  6. 2018Anaphora and Coreference Resolution: A Reviewcited by 31
  7. 2021ADePT: Auto-encoder based Differentially Private Text Transformationcited by 31
  8. 2014C-sanitized: a privacy model for document redaction and sanitizationcited by 27
  9. 2021When differential privacy meets NLP: The devil is in the detailcited by 25
  10. 2021No Intruder, no Validity: Evaluation Criteria for Privacy-Preserving Text Anonymizationcited by 13
  11. 2021A Privacy-Preserving Approach to Extraction of Personal Information through Automatic Annotation and Federated Learningcited by 12
  12. 2021De-identification of Privacy-related Entities in Job Postingscited by 12
  13. 2017Toward sensitive document release with privacy guaranteescited by 6
  14. 2018A survey of automatic de-identification of longitudinal clinical narrativescited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 13; 2024: 19; 2025: 26; 2026: 32.

94 papers.

Sort:

By area:

  1. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  2. 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 278
  3. 2024Clio: Privacy-Preserving Insights into Real-World AI Usecited by 66
  4. 2023Reducing Privacy Risks in Online Self-Disclosures with Language Modelscited by 47
  5. 2024LLM-PBE: Assessing Data Privacy in Large Language Modelscited by 38
  6. 2024Privacy-Preserving Instructions for Aligning Large Language Modelscited by 34
  7. 2024Large Language Models are Advanced Anonymizerscited by 22
  8. 2024IncogniText: Privacy-enhancing Conditional Text Anonymization via LLM-based Private Attribute Randomizationcited by 21
  9. 2023Differentially Private Natural Language Models: Recent Advances and Future Directionscited by 20
  10. 2023Privacy- and Utility-Preserving NLP with Anonymized Data: A case study of Pseudonymizationcited by 18
  11. 2025A False Sense of Privacy: Evaluating Textual Data Sanitization Beyond Surface-level Privacy Leakagecited by 15
  12. 2025Tokens for Learning, Tokens for Unlearning: Mitigating Membership Inference Attacks in Large Language Models via Dual-Purpose Trainingcited by 14
38 more
  1. 2022Unsupervised Text Deidentificationcited by 11
  2. 2022Privacy-Preserving Models for Legal Natural Language Processingcited by 10
  3. 2023Beyond Document Page Classification: Design, Datasets, and Challengescited by 10
  4. 2023Text Sanitization Beyond Specific Domains: Zero-Shot Redaction & Substitution with Large Language Modelscited by 10
  5. 2025InvisibleInk: High-Utility and Low-Cost Text Generation with Differential Privacycited by 10
  6. 2025LLM-as-a-Judge for Privacy Evaluation? Exploring the Alignment of Human and LLM Perceptions of Privacy in Textual Datacited by 10
  7. 2023Does fine-tuning GPT-3 with the OpenAI API leak personally-identifiable information?cited by 9
  8. 2023Gradient-Free Privacy Leakage in Federated Language Models through Selective Weight Tamperingcited by 9
  9. 2024Robust Utility-Preserving Text Anonymization Based on Large Language Modelscited by 9
  10. 2025SOFT: Selective Data Obfuscation for Protecting LLM Fine-tuning against Membership Inference Attackscited by 9
  11. 2022Bootstrapping Text Anonymization Models with Distant Supervisioncited by 8
  12. 2024NAP^2: A Benchmark for Naturalness and Privacy-Preserving Text Rewriting by Learning from Humancited by 8
  13. 2024Casper: Prompt Sanitization for Protecting User Privacy in Web-Based Large Language Modelscited by 8
  14. 2022Addressing Resource and Privacy Constraints in Semantic Parsing Through Data Augmentationcited by 7
  15. 2024Unlocking the Potential of Large Language Models for Clinical Text Anonymization: A Comparative Studycited by 7
  16. 2025PII-Bench: Evaluating Query-Aware Privacy Protection Systemscited by 7
  17. 2025SynthTextEval: Synthetic Text Data Generation and Evaluation for High-Stakes Domainscited by 7
  18. 2023Informed Named Entity Recognition Decoding for Generative Language Modelscited by 6
  19. 2025Automated Privacy Information Annotation in Large Language Model Interactionscited by 6
  20. 2024Benchmarking Advanced Text Anonymisation Methods: A Comparative Study on Novel and Traditional Approachescited by 5
  21. 2025DP-Fusion: Token-Level Differentially Private Inference for Large Language Modelscited by 5
  22. 2025A Survey on Current Trends and Recent Advances in Text Anonymizationcited by 5
  23. 2025How do we measure privacy in text? A survey of text anonymization metricscited by 5
  24. 2023Neural Text Sanitization with Privacy Risk Indicators: An Empirical Analysiscited by 4
  25. 2024Reconstruction of Differentially Private Text Sanitization via Large Language Modelscited by 4
  26. 2024DIRI: Adversarial Patient Reidentification with Large Language Models for Evaluating Clinical Text Anonymizationcited by 4
  27. 2025Towards Harnessing the Collaborative Power of Large and Small Models for Domain Taskscited by 4
  28. 2025Can Large Language Models Really Recognize Your Name?cited by 4
  29. 2025Self-Refining Language Model Anonymizers via Adversarial Distillationcited by 4
  30. 2023Crowdsourcing on Sensitive Data with Privacy-Preserving Text Rewritingcited by 3
  31. 2024Cloaked Classifiers: Pseudonymization Strategies on Sensitive Classification Taskscited by 3
  32. 2024Private Language Models via Truncated Laplacian Mechanismcited by 3
  33. 2025Re-identification of De-identified Documents with Autoregressive Infillingcited by 3
  34. 2026Privasis: Synthesizing the Largest "Public" Private Dataset from Scratchcited by 3
  35. 2026CAPID: Context-Aware PII Detection for Question-Answering Systemscited by 3
  36. 2026Need to Know: Contextual-Integrity-Grounded Query Rewriting for Privacy-Conscious LLM Delegationcited by 3
  37. 2026REDACT: A Systematically Controlled Multilingual Benchmark for Personal Information Detectioncited by 3
  38. 2024How Private are Language Models in Abstractive Summarization?cited by 2

50 most cited of 94 All 94 (JSON)