mlsanityData to 13 Oct 2026

Sentence-level Privacy for Document Embeddings

arXiv 2205.04605 ↗First posted 10 May 2022cs.LG cs.CLcited by 16

Not on the map. Why?

Abstract (arXiv)

User language data can contain highly sensitive personal content. As such, it is imperative to offer users a strong and interpretable privacy guarantee when learning from their data. In this work, we propose SentDP: pure local differential privacy at the sentence level for a single user document. We propose a novel technique, DeepCandidate, that combines concepts from robust statistics and language modeling to produce high-dimensional, general-purpose ε-SentDP document embeddings. This guarantees that any single sentence in a document can be substituted with any other sentence while keeping the embedding ε-indistinguishable. Our experiments indicate that these private document embeddings are useful for downstream tasks like sentiment analysis and topic classification and even outperform baseline methods with weaker guarantees like word-level Metric DP.

Key facts. Cited by 16 papers in the map (data through 13 Oct 2026). Builds on 24 of its 53 references that are in the map. First posted on arXiv 10 May 2022.

Builds on

24 of 53 references.

References · 24

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,322
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,107
  4. 2019Advances and Open Problems in Federated Learningcited by 3,040
  5. 2016Deep Learning with Differential Privacycited by 2,718
  6. 2020Extracting Training Data from Large Language Modelscited by 2,181
  7. 2016Enriching Word Vectors with Subword Informationcited by 1,581
  8. 2019Publicly Available Clinical BERT Embeddingscited by 999
  9. 2019Multi-Task Deep Neural Networks for Natural Language Understandingcited by 350
  10. 2019Differential Privacy Has Disparate Impact on Model Accuracycited by 316
  11. 2020Information Leakage in Embedding Modelscited by 203
  12. 2020Pre-training is a Hot Topic: Contextualized Document Embeddings Improve Topic Coherencecited by 121
12 more
  1. 2019Privacy- and Utility-Preserving Textual Analysis via Calibrated Multivariate Perturbationscited by 118
  2. 2018Privately Learning High-Dimensional Distributionscited by 104
  3. 2020On the Sentence Embeddings from Pre-trained Language Modelscited by 83
  4. 2021Differential Privacy for Text Analytics via Natural Text Sanitizationcited by 76
  5. 2018Linguistically-Informed Self-Attention for Semantic Role Labelingcited by 71
  6. 2020Differentially Private Language Models Benefit from Public Pre-trainingcited by 46
  7. 2021TEM: High Utility Metric Differential Privacy on Textcited by 43
  8. 2020TextHide: Tackling Data Privacy in Language Understanding Taskscited by 33
  9. 2019Private Center Points and Learning of Halfspacescited by 14
  10. 2021Differentially Private Imaging via Latent Space Manipulationcited by 13
  11. 2019Better Word Embeddings by Disentangling Contextual n-Gram Informationcited by 12
  12. 2020When does the Tukey median work?cited by 5

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2022: 1; 2023: 5; 2024: 5; 2025: 2; 2026: 3.

16 papers.

Sort:

By area:

  1. 2022Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipecited by 108
  2. 2023DP-Forward: Fine-tuning and Inference on Language Models with Differential Privacy in Forward Passcited by 66
  3. 2023Training Data Extraction From Pre-trained Language Models: A Surveycited by 44
  4. 2024Privacy Preserving Prompt Engineering: A Surveycited by 25
  5. 2023Differentially Private Natural Language Models: Recent Advances and Future Directionscited by 20
  6. 2024The Fire Thief Is Also the Keeper: Balancing Usability and Privacy in Promptscited by 13
  7. 20241-Diffractor: Efficient and Utility-Preserving Text Obfuscation Leveraging Word-Level Metric Differential Privacycited by 10
  8. 2024Just Rewrite It Again: A Post-Processing Method for Enhanced Semantic Similarity and Privacy Preservation of Differentially Private Rewritten Textcited by 9
  9. 2025Spend Your Budget Wisely: Towards an Intelligent Distribution of the Privacy Budget in Differentially Private Text Rewritingcited by 5
  10. 2023LatticeGen: A Cooperative Framework which Hides Generated Text in a Lattice for Privacy-Aware Generation on Cloudcited by 2
  11. 2023PrIeD-KIE: Towards Privacy Preserved Document Key Information Extractioncited by 2
  12. 2026Differential Privacy for Transformer Embeddings of Text with Nonparametric Variational Information Bottleneckcited by 2
4 more
  1. 2025BinaryShield: Cross-Service Threat Intelligence in LLM Services using Privacy-Preserving Fingerprintscited by 1
  2. 2026Demystifying the Privacy-Utility Trade-off in LLM Interactionscited by 1
  3. 2024IDT: Dual-Task Adversarial Attacks for Privacy Protectioncited by 0
  4. 2026DeepInvert: Semi-Supervised Embedding Inversion Against Obfuscated Language Modelscited by 0