mlsanityData to 14 Oct 2026

PrivBench: A Holistic and Modular Benchmarking Platform for Evaluating Text-to-Text Privatization

arXiv 2608.29624 ↗First posted 30 Aug 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Natural Language Processing methods have enabled novel solutions and advances in the field of privacy, particularly in the sub-domain of text-to-text privatization, where the goal is to transform a sensitive input text into a privatized output by ideally masking (in)directly identifiable or otherwise private information. The evaluation of text-to-text privatization, however, is not straightforward, and the extant literature has utilized a myriad of techniques and metrics to quantify the privacy-preserving capabilities of privatization methods. Seeking to unify the evaluation of text-to-text privatization, we introduce PrivBench, a holistic and modular benchmarking platform for researchers and practitioners working on text privatization. PrivBench is holistic in that it evaluates privatization on a series of defined desiderata, which are structured into modules. PrivBench is not only modular but also extensible, allowing for future updates and benchmark versions. PrivBench is user-centered and promotes competition via real-time evaluation and a live public leaderboard. The platform is free to use and openly accessible at https://privbench.com/.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 105 of its 205 references that are in the map. First posted on arXiv 30 Aug 2026.

Builds on

105 of 205 references.

References · 105

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  2. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  3. 2016Pointer Sentinel Mixture Modelscited by 3,096
  4. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  5. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,288
  6. 2021Differentially Private Fine-tuning of Language Modelscited by 400
  7. 2024PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Actioncited by 171
  8. 2024OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learningcited by 128
  9. 2018Style Transfer Through Back-Translationcited by 124
  10. 2019Privacy- and Utility-Preserving Textual Analysis via Calibrated Multivariate Perturbationscited by 119
  11. 2022Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipecited by 108
  12. 2023Privacy-Preserving In-Context Learning with Differentially Private Few-Shot Generationcited by 96
38 more
  1. 2021Differential Privacy for Text Analytics via Natural Text Sanitizationcited by 77
  2. 2024Differentially Private Synthetic Data via Foundation Model APIs 2: Textcited by 76
  3. 2018Towards Robust and Privacy-preserving Text Representationscited by 75
  4. 2022Differentially Private Language Models for Secure Data Sharingcited by 68
  5. 2022The Limits of Word Level Differential Privacycited by 63
  6. 2023Locally Differentially Private Document Generation Using Zero Shot Promptingcited by 63
  7. 2021Selective Differential Privacy for Language Modelingcited by 62
  8. 2022THE-X: Privacy-Preserving Transformer Inference with Homomorphic Encryptioncited by 57
  9. 2021Natural Language Understanding with Privacy-Preserving BERTcited by 55
  10. 2018Generalised Differential Privacy for Text Document Processingcited by 52
  11. 2022A Customized Text Sanitization Mechanism with Differential Privacycited by 52
  12. 2019Leveraging Hierarchical Representations for Preserving Privacy and Utility in Textcited by 50
  13. 2020Differentially Private Language Models Benefit from Public Pre-trainingcited by 47
  14. 2021TEM: High Utility Metric Differential Privacy on Textcited by 43
  15. 2020A Differentially Private Text Perturbation Method Using a Regularized Mahalanobis Metriccited by 42
  16. 2022Attribution and Obfuscation of Neural Text Authorship: A Data Mining Perspectivecited by 41
  17. 2023PrivLM-Bench: A Multi-level Privacy Evaluation Benchmark for Language Modelscited by 37
  18. 2022Privacy Adhering Machine Un-learning in NLPcited by 36
  19. 2019ER-AE: Differentially Private Text Generation for Authorship Anonymizationcited by 35
  20. 2024On the Privacy Risk of In-context Learningcited by 32
  21. 2021ADePT: Auto-encoder based Differentially Private Text Transformationcited by 31
  22. 2022Submix: Practical Private Prediction for Large-Scale Language Modelscited by 31
  23. 2022Text Style Transfer for Bias Mitigation using Masked Language Modelingcited by 28
  24. 2022DP-Rewrite: Towards Reproducibility and Transparency in Differentially Private Text Rewritingcited by 28
  25. 2023Quantifying Association Capabilities of Large Language Models and Its Implications on Privacy Leakagecited by 28
  26. 2024PrivacyRestore: Privacy-Preserving Inference in Large Language Models via Privacy Removal and Restorationcited by 28
  27. 2014C-sanitized: a privacy model for document redaction and sanitizationcited by 27
  28. 2023MoPe: Model Perturbation-based Privacy Attacks on Language Modelscited by 25
  29. 2024DP-MLM: Differentially Private Text Rewriting Using Masked Language Modelscited by 24
  30. 2024IncogniText: Privacy-enhancing Conditional Text Anonymization via LLM-based Private Attribute Randomizationcited by 23
  31. 2021CAPE: Context-Aware Private Embeddings for Private Language Learningcited by 22
  32. 2018SynTF: Synthetic and Differentially Private Term Frequency Vectors for Privacy-Preserving Text Miningcited by 21
  33. 2024Comparing Retrieval-Augmentation and Parameter-Efficient Fine-Tuning for Privacy-Preserving Personalization of Large Language Modelscited by 21
  34. 2021Privacy-Preserving Graph Convolutional Networks for Text Classificationcited by 19
  35. 2023Privacy- and Utility-Preserving NLP with Anonymized Data: A case study of Pseudonymizationcited by 18
  36. 2024Differentially Private Knowledge Distillation via Synthetic Text Generationcited by 18
  37. 2021On a Utilitarian Approach to Privacy Preserving Text Generationcited by 17
  38. 2023Guiding Text-to-Text Privatization by Syntaxcited by 14

50 most cited of 105 All 105 (JSON)

Built on by

Not cited yet.