mlsanityData to 7 Oct 2026

CAPE: Context-Aware Private Embeddings for Private Language Learning

arXiv 2108.12318 ↗First posted 27 Aug 2021cs.CL cs.LGcited by 22

Not on the map. Why?

Abstract (arXiv)

Deep learning-based language models have achieved state-of-the-art results in a number of applications including sentiment analysis, topic labelling, intent classification and others. Obtaining text representations or embeddings using these models presents the possibility of encoding personally identifiable information learned from language and context cues that may present a risk to reputation or privacy. To ameliorate these issues, we propose Context-Aware Private Embeddings (CAPE), a novel approach which preserves privacy during training of embeddings. To maintain the privacy of text representations, CAPE applies calibrated noise through differential privacy, preserving the encoded semantic links while obscuring sensitive information. In addition, CAPE employs an adversarial training regime that obscures identified private variables. Experimental results demonstrate that the proposed approach reduces private information leakage better than either single intervention.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 25 of its 47 references that are in the map. First posted on arXiv 27 Aug 2021.

Builds on

25 of 47 references.

References · 25

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  5. 2020Extracting Training Data from Large Language Modelscited by 2,144
  6. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  7. 2019The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networkscited by 241
  8. 2020Information Leakage in Embedding Modelscited by 199
  9. 2019Stolen Memories: Leveraging Model Memorization for Calibrated White-Box Membership Inferencecited by 168
  10. 2017Machine Learning Models that Remember Too Muchcited by 165
  11. 2019Privacy- and Utility-Preserving Textual Analysis via Calibrated Multivariate Perturbationscited by 115
  12. 2021Debiasing Pre-trained Contextualised Embeddingscited by 107
13 more
  1. 2019Toward Gender-Inclusive Coreference Resolutioncited by 84
  2. 2018Embedding Text in Hyperbolic Spacescited by 76
  3. 2018Towards Robust and Privacy-preserving Text Representationscited by 75
  4. 2020Differentially Private Representation for NLP: Formal Guarantee and An Empirical Study on Privacy and Fairnesscited by 66
  5. 2018Privacy-preserving Neural Representations of Textcited by 54
  6. 2018Generalised Differential Privacy for Text Document Processingcited by 52
  7. 2018Towards Demystifying Membership Inference Attackscited by 39
  8. 2015Heterogeneous Differential Privacycited by 30
  9. 2020Compositional Demographic Word Embeddingscited by 20
  10. 2019I Am Not What I Write: Privacy Preserving Text Representation Learningcited by 12
  11. 2019Adversarial Learning of Privacy-Preserving Text Representations for De-Identification of Medical Recordscited by 11
  12. 2019dpUGC: Learn Differentially Private Representation for User Generated Contentscited by 5
  13. 2019Scalable Differential Privacy with Certified Robustness in Adversarial Learningcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 4; 2024: 7; 2025: 5; 2026: 4.

22 papers.

Sort:

By area:

  1. 2023DEPN: Detecting and Editing Privacy Neurons in Pretrained Language Modelscited by 75
  2. 2023Privacy-Preserving Parameter-Efficient Fine-Tuning for Large Language Model Servicescited by 43
  3. 2024Privacy Preserving Prompt Engineering: A Surveycited by 24
  4. 2023Differentially Private Natural Language Models: Recent Advances and Future Directionscited by 20
  5. 2024The Fire Thief Is Also the Keeper: Balancing Usability and Privacy in Promptscited by 13
  6. 2022You Are What You Write: Preserving Privacy in the Era of Large Language Modelscited by 11
  7. 2024A Comparative Analysis of Word-Level Metric Differential Privacy: Benchmarking The Privacy-Utility Trade-offcited by 11
  8. 2022Fair NLP Models with Differentially Private Text Encoderscited by 9
  9. 2024Privacy-Preserving Language Model Inference with Instance Obfuscationcited by 6
  10. 2025Towards Harnessing the Collaborative Power of Large and Small Models for Domain Taskscited by 4
  11. 2024Private Language Models via Truncated Laplacian Mechanismcited by 3
  12. 2023PrIeD-KIE: Towards Privacy Preserved Document Key Information Extractioncited by 2
10 more
  1. 2024Enhancing Text-to-SQL Capabilities of Large Language Models via Domain Database Knowledge Injectioncited by 2
  2. 2025Towards Privacy-aware Mental Health AI Models: Advances, Challenges, and Opportunitiescited by 2
  3. 2025Anti-adversarial Learning: Desensitizing Prompts for Large Language Modelscited by 2
  4. 2024Protecting Privacy in Classifiers by Token Manipulationcited by 1
  5. 2025Investigating User Perspectives on Differentially Private Text Privatizationcited by 1
  6. 2025DP-DocLDM: Differentially Private Document Image Generation using Latent Diffusion Modelscited by 1
  7. 2026Demystifying the Privacy-Utility Trade-off in LLM Interactionscited by 1
  8. 2026Not All Entities are Created Equal: A Dynamic Anonymization Framework for Privacy-Preserving RAGcited by 0
  9. 2026DeepInvert: Semi-Supervised Embedding Inversion Against Obfuscated Language Modelscited by 0
  10. 2026PrivBench: A Holistic and Modular Benchmarking Platform for Evaluating Text-to-Text Privatizationcited by 0