# Privacy- and Utility-Preserving NLP with Anonymized Data: A case study of Pseudonymization

arXiv: https://arxiv.org/abs/2306.05561 · HTML: https://mlsanity.com/paper/2306.05561/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 41 references that are in the map.
- First posted on arXiv 8 Jun 2023.

## Builds on (most cited first, 11 of 11)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond](https://mlsanity.com/paper/1602.06023/) (2016)
- [DeID-GPT: Zero-shot Medical Text De-Identification by GPT-4](https://mlsanity.com/paper/2303.11032/) (2023)
- [The Text Anonymization Benchmark (TAB): A Dedicated Corpus and Evaluation Framework for Text Anonymization](https://mlsanity.com/paper/2202.00443/) (2022)
- [De-identification of Patient Notes with Recurrent Neural Networks](https://mlsanity.com/paper/1606.03475/) (2016)
- [Differential Privacy for Text Analytics via Natural Text Sanitization](https://mlsanity.com/paper/2106.01221/) (2021)
- [C-sanitized: a privacy model for document redaction and sanitization](https://mlsanity.com/paper/1406.4285/) (2014)
- [Bootstrapping Text Anonymization Models with Distant Supervision](https://mlsanity.com/paper/2205.06895/) (2022)

## Built on by (most cited first, 18 of 18)

- [Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Models](https://mlsanity.com/paper/2308.16149/) (2023)
- [Evaluating Differentially Private Synthetic Data Generation in High-Stakes Domains](https://mlsanity.com/paper/2410.08327/) (2024)
- [Robust Utility-Preserving Text Anonymization Based on Large Language Models](https://mlsanity.com/paper/2407.11770/) (2024)
- [Neural Text Sanitization with Privacy Risk Indicators: An Empirical Analysis](https://mlsanity.com/paper/2310.14312/) (2023)
- [A Survey on Current Trends and Recent Advances in Text Anonymization](https://mlsanity.com/paper/2508.21587/) (2025)
- [A General Pseudonymization Framework for Cloud-Based LLMs: Replacing Privacy Information in Controlled Text Generation](https://mlsanity.com/paper/2502.15233/) (2025)
- [PATCH: Mitigating PII Leakage in Language Models with Privacy-Aware Targeted Circuit PatcHing](https://mlsanity.com/paper/2510.07452/) (2025)
- [User Perceptions vs. Proxy LLM Judges: Privacy and Helpfulness in LLM Responses to Privacy-Sensitive Scenarios](https://mlsanity.com/paper/2510.20721/) (2025)
- [Adaptive Text Anonymization: Learning Privacy-Utility Trade-offs via Prompt Optimization](https://mlsanity.com/paper/2602.20743/) (2026)
- [Current State in Privacy-Preserving Text Preprocessing for Domain-Agnostic NLP](https://mlsanity.com/paper/2508.03204/) (2025)
- [A Study of Privacy-preserving Language Modeling Approaches](https://mlsanity.com/paper/2508.15421/) (2025)
- [An Artificial Intelligence Value at Risk Approach: Metrics and Models](https://mlsanity.com/paper/2509.18394/) (2025)
- [Say Something Else: Rethinking Contextual Privacy as Information Sufficiency](https://mlsanity.com/paper/2604.06409/) (2026)
- [Surrogate Substitution Preserves PHI Detectability: A Multi-Detector Equivalence Study](https://mlsanity.com/paper/2608.03172/) (2026)
- [From Interpretability to Control: Insights from Six Years of the TrustNLP Workshop](https://mlsanity.com/paper/2608.11171/) (2026)
- [No PUN Intended: Plausible Unknown Names for Person-Centred LLM Evaluation](https://mlsanity.com/paper/2608.21206/) (2026)
- [PrivBench: A Holistic and Modular Benchmarking Platform for Evaluating Text-to-Text Privatization](https://mlsanity.com/paper/2608.29624/) (2026)
- [On the Impact of Anonymization on the Performance of Large Language Models](https://mlsanity.com/paper/2609.11335/) (2026)

## Abstract (arXiv)

This work investigates the effectiveness of different pseudonymization techniques, ranging from rule-based substitutions to using pre-trained Large Language Models (LLMs), on a variety of datasets and models used for two widely used NLP tasks: text classification and summarization. Our work provides crucial insights into the gaps between original and anonymized data (focusing on the pseudonymization technique) and model quality and fosters future research into higher-quality anonymization techniques to better balance the trade-offs between data protection and utility preservation. We make our code, pseudonymized datasets, and downstream models publicly available