mlsanityData to 14 Oct 2026

A Neighbourhood-Aware Differential Privacy Mechanism for Static Word Embeddings

arXiv 2309.10551 ↗First posted 19 Sep 2023cs.LG cs.AI cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

We propose a Neighbourhood-Aware Differential Privacy (NADP) mechanism considering the neighbourhood of a word in a pretrained static word embedding space to determine the minimal amount of noise required to guarantee a specified privacy level. We first construct a nearest neighbour graph over the words using their embeddings, and factorise it into a set of connected components (i.e. neighbourhoods). We then separately apply different levels of Gaussian noise to the words in each neighbourhood, determined by the set of words in that neighbourhood. Experiments show that our proposed NADP mechanism consistently outperforms multiple previously proposed DP mechanisms such as Laplacian, Gaussian, and Mahalanobis in multiple downstream tasks, while guaranteeing higher levels of privacy.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 28 of its 52 references that are in the map. First posted on arXiv 19 Sep 2023.

Builds on

28 of 52 references.

References · 28

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  4. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  5. 2016Deep Learning with Differential Privacycited by 2,728
  6. 2016Enriching Word Vectors with Subword Informationcited by 1,583
  7. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 1,001
  8. 2019Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Themcited by 268
  9. 2018Improving the Gaussian Mechanism for Differential Privacy: Analytical Calibration and Optimal Denoisingcited by 218
  10. 2014SimLex-999: Evaluating Semantic Models with (Genuine) Similarity Estimationcited by 172
  11. 2018Learning Gender-Neutral Word Embeddingscited by 169
  12. 2019Privacy- and Utility-Preserving Textual Analysis via Calibrated Multivariate Perturbationscited by 119
16 more
  1. 2018Towards Robust and Privacy-preserving Text Representationscited by 75
  2. 2020Differentially Private Representation for NLP: Formal Guarantee and An Empirical Study on Privacy and Fairnesscited by 67
  3. 2019Gender-preserving Debiasing for Pre-trained Word Embeddingscited by 61
  4. 2016SimVerb-3500: A Large-Scale Evaluation Set of Verb Similaritycited by 44
  5. 2020A Differentially Private Text Perturbation Method Using a Regularized Mahalanobis Metriccited by 42
  6. 2016Privacy Odometers and Filters: Pay-as-you-Go Compositioncited by 33
  7. 2021ADePT: Auto-encoder based Differentially Private Text Transformationcited by 31
  8. 2020Towards Differentially Private Text Representationscited by 29
  9. 2021When differential privacy meets NLP: The devil is in the detailcited by 25
  10. 2021Dictionary-based Debiasing of Pre-trained Word Embeddingscited by 24
  11. 2021Obtaining Better Static Word Embeddings Using Contextual Embedding Modelscited by 18
  12. 2019I Am Not What I Write: Privacy Preserving Text Representation Learningcited by 12
  13. 2022How reparametrization trick broke differentially-private text representation learningcited by 10
  14. 2022A Survey on Word Meta-Embedding Learningcited by 7
  15. 2017Using k-way Co-occurrences for Learning Word Embeddingscited by 5
  16. 2022Learning Meta Word Embeddings by Unsupervised Weighted Concatenation of Source Embeddingscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 1.

4 papers.

  1. 2024Just Rewrite It Again: A Post-Processing Method for Enhanced Semantic Similarity and Privacy Preservation of Differentially Private Rewritten Textcited by 9
  2. 2025Investigating User Perspectives on Differentially Private Text Privatizationcited by 1
  3. 2025BinaryShield: Cross-Service Threat Intelligence in LLM Services using Privacy-Preserving Fingerprintscited by 1
  4. 2026A Constrained Optimization Perspective of Unrolled Transformerscited by 0