# Tau-Eval: A Unified Evaluation Framework for Useful and Private Text Anonymization

arXiv: https://arxiv.org/abs/2506.05979 · HTML: https://mlsanity.com/paper/2506.05979/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 54 references that are in the map.
- First posted on arXiv 6 Jun 2025.

## Builds on (most cited first, 26 of 26)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Neural Network Acceptability Judgments](https://mlsanity.com/paper/1805.12471/) (2018)
- [Adversarial NLI: A New Benchmark for Natural Language Understanding](https://mlsanity.com/paper/1910.14599/) (2019)
- [Beyond English-Centric Multilingual Machine Translation](https://mlsanity.com/paper/2010.11125/) (2020)
- [Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference](https://mlsanity.com/paper/2412.13663/) (2024)
- [Datasets: A Community Library for Natural Language Processing](https://mlsanity.com/paper/2109.02846/) (2021)
- [ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission](https://mlsanity.com/paper/1904.05342/) (2019)
- [Dynabench: Rethinking Benchmarking in NLP](https://mlsanity.com/paper/2104.14337/) (2021)
- [Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection](https://mlsanity.com/paper/2012.15761/) (2020)
- [WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation](https://mlsanity.com/paper/2201.05955/) (2022)
- [Lessons from Natural Language Inference in the Clinical Domain](https://mlsanity.com/paper/1808.06752/) (2018)
- [XLM-T: Multilingual Language Models in Twitter for Sentiment Analysis and Beyond](https://mlsanity.com/paper/2104.12250/) (2021)
- [DynaSent: A Dynamic Benchmark for Sentiment Analysis](https://mlsanity.com/paper/2012.15349/) (2020)
- [Keep it Simple: Unsupervised Simplification of Multi-Paragraph Text](https://mlsanity.com/paper/2107.03444/) (2021)
- [Evaluate & Evaluation on the Hub: Better Best Practices for Data and Model Measurements](https://mlsanity.com/paper/2210.01970/) (2022)
- [StyleRemix: Interpretable Authorship Obfuscation via Distillation and Perturbation of Style Elements](https://mlsanity.com/paper/2408.15666/) (2024)
- [Keep It Private: Unsupervised Privatization of Online Text](https://mlsanity.com/paper/2405.10260/) (2024)
- [CamemBERT-bio: Leveraging Continual Pre-training for Cost-Effective Models on French Biomedical Data](https://mlsanity.com/paper/2306.15550/) (2023)
- [TAROT: Task-Oriented Authorship Obfuscation Using Policy Optimization Methods](https://mlsanity.com/paper/2407.21630/) (2024)
- [Closing the Gap: Joint De-Identification and Concept Extraction in the Clinical Domain](https://mlsanity.com/paper/2005.09397/) (2020)
- [Cloaked Classifiers: Pseudonymization Strategies on Sensitive Classification Tasks](https://mlsanity.com/paper/2406.17875/) (2024)

## Built on by (most cited first, 4 of 4)

- [Adaptive Text Anonymization: Learning Privacy-Utility Trade-offs via Prompt Optimization](https://mlsanity.com/paper/2602.20743/) (2026)
- [Distilling Human-Aligned Privacy Sensitivity Assessment from Large Language Models](https://mlsanity.com/paper/2603.29497/) (2026)
- [Introducing the Privacy-HSD Trade-off: Hate Speech Detection, but not at the Cost of Privacy](https://mlsanity.com/paper/2608.19006/) (2026)
- [PrivBench: A Holistic and Modular Benchmarking Platform for Evaluating Text-to-Text Privatization](https://mlsanity.com/paper/2608.29624/) (2026)

## Abstract (arXiv)

Text anonymization is the process of removing or obfuscating information from textual data to protect the privacy of individuals. This process inherently involves a complex trade-off between privacy protection and information preservation, where stringent anonymization methods can significantly impact the text's utility for downstream applications. Evaluating the effectiveness of text anonymization proves challenging from both privacy and utility perspectives, as there is no universal benchmark that can comprehensively assess anonymization techniques across diverse, and sometimes contradictory contexts. We present Tau-Eval, an open-source framework for benchmarking text anonymization methods through the lens of privacy and utility task sensitivity. A Python library, code, documentation and tutorials are publicly available.