mlsanityData to 13 Oct 2026

Towards Robust and Privacy-preserving Text Representations

arXiv 1805.06093 ↗First posted 16 May 2018cs.CLcited by 75

Not on the map. Why?

Abstract (arXiv)

Written text often provides sufficient clues to identify the author, their gender, age, and other important attributes. Consequently, the authorship of training and evaluation corpora can have unforeseen impacts, including differing model performance for different user groups, as well as privacy implications. In this paper, we propose an approach to explicitly obscure important author characteristics at training time, such that representations learned are invariant to these attributes. Evaluating on two tasks, we show that this leads to increased privacy in the learned representations, as well as more robust models to varying evaluation conditions, including out-of-domain corpora.

Key facts. Cited by 75 papers in the map (data through 13 Oct 2026). First posted on arXiv 16 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 23; 2022: 27; 2023: 7; 2024: 13; 2025: 2; 2026: 3.

75 papers.

Sort:

By area:

  1. 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 314
  2. 2021Gender Bias in Machine Translationcited by 116
  3. 2022Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Datasetcited by 113
  4. 2021Debiasing Pre-trained Contextualised Embeddingscited by 107
  5. 2022The Text Anonymization Benchmark (TAB): A Dedicated Corpus and Evaluation Framework for Text Anonymizationcited by 95
  6. 2021A Survey on Gender Bias in Natural Language Processingcited by 89
  7. 2021Differential Privacy for Text Analytics via Natural Text Sanitizationcited by 76
  8. 2021Natural Language Understanding with Privacy-Preserving BERTcited by 55
  9. 2021Diverse Adversaries for Mitigating Bias in Trainingcited by 48
  10. 2021NeuTral Rewriter: A Rule-Based and Neural Approach to Automatic Rewriting into Gender-Neutral Alternativescited by 46
  11. 2022Towards Identifying Social Bias in Dialog Systems: Frame, Datasets, and Benchmarkscited by 36
  12. 2021Balancing out Bias: Achieving Fairness Through Balanced Trainingcited by 34
38 more
  1. 2021Contrastive Learning for Fair Representationscited by 34
  2. 2022Gold Doesn't Always Glitter: Spectral Removal of Linear and Nonlinear Guarded Attribute Informationcited by 33
  3. 2022Submix: Practical Private Prediction for Large-Scale Language Modelscited by 31
  4. 2021Privacy Regularization: Joint Privacy-Utility Optimization in Language Modelscited by 30
  5. 2021Evaluating Debiasing Techniques for Intersectional Biasescited by 30
  6. 2022Choose Your Lenses: Flaws in Gender Bias Evaluationcited by 29
  7. 2021Benchmarking Differential Privacy and Federated Learning for BERT Modelscited by 28
  8. 2023Survey of Social Bias in Vision-Language Modelscited by 26
  9. 2021Dictionary-based Debiasing of Pre-trained Word Embeddingscited by 24
  10. 2021CAPE: Context-Aware Private Embeddings for Private Language Learningcited by 22
  11. 2022Optimising Equal Opportunity Fairness in Model Trainingcited by 22
  12. 2021Privacy-Preserving Graph Convolutional Networks for Text Classificationcited by 19
  13. 2021Fairness-aware Class Imbalanced Learningcited by 19
  14. 2022How to keep text private? A systematic review of deep learning methods for privacy-preserving natural language processingcited by 16
  15. 2021Generative Adversarial Networks: A Survey Towards Private and Secure Applicationscited by 15
  16. 2022Learning Fair Representations via Rate-Distortion Maximizationcited by 15
  17. 2021Adversarial Scrubbing of Demographic Information for Text Classificationcited by 14
  18. 2021Adversarial Stylometry in the Wild: Transferable Lexical Substitution Attacks on Author Profilingcited by 13
  19. 2022You Are What You Write: Preserving Privacy in the Era of Large Language Modelscited by 12
  20. 2022You Don't Know My Favorite Color: Preventing Dialogue Representations from Revealing Speakers' Private Personascited by 11
  21. 2023Fair Enough: Standardizing Evaluation and Model Selection for Fairness Research in NLPcited by 11
  22. 2024Model Inversion Attacks: A Survey of Approaches and Countermeasurescited by 11
  23. 2021Mitigating the Position Bias of Transformer Models in Passage Re-Rankingcited by 10
  24. 2022How reparametrization trick broke differentially-private text representation learningcited by 10
  25. 2024Generative AI for Requirements Engineering: A Systematic Literature Reviewcited by 10
  26. 2022Fair NLP Models with Differentially Private Text Encoderscited by 9
  27. 2022Privacy-Preserving Text Classification on BERT Embeddings with Homomorphic Encryptioncited by 9
  28. 2023Learning to Generate Text in Arbitrary Writing Stylescited by 9
  29. 2021Killing One Bird with Two Stones: Model Extraction and Attribute Inference Attacks against BERT-based APIscited by 8
  30. 2022Towards Equal Opportunity Fairness through Adversarial Learningcited by 8
  31. 2022FairGrad: Fairness Aware Gradient Descentcited by 8
  32. 2024NAP^2: A Benchmark for Naturalness and Privacy-Preserving Text Rewriting by Learning from Humancited by 8
  33. 2022SocioProbe: What, When, and Where Language Models Learn about Sociodemographicscited by 7
  34. 2024Privacy-Preserving Language Model Inference with Instance Obfuscationcited by 7
  35. 2022Gender Bias in Meta-Embeddingscited by 6
  36. 2023Erasure of Unaligned Attributes from Neural Representationscited by 6
  37. 2023DualFair: Fair Representation Learning at Both Group and Individual Levels via Contrastive Self-supervisioncited by 6
  38. 2021Unsupervised Information Obfuscation for Split Inference of Neural Networkscited by 5

50 most cited of 75 All 75 (JSON)