# How reparametrization trick broke differentially-private text representation learning

arXiv: https://arxiv.org/abs/2202.12138 · HTML: https://mlsanity.com/paper/2202.12138/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 23 references that are in the map.
- First posted on arXiv 24 Feb 2022.

## Builds on (most cited first, 9 of 9)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Renyi Differential Privacy](https://mlsanity.com/paper/1702.07476/) (2017)
- [Towards Robust and Privacy-preserving Text Representations](https://mlsanity.com/paper/1805.06093/) (2018)
- [Detecting Violations of Differential Privacy](https://mlsanity.com/paper/1805.10277/) (2018)
- [ADePT: Auto-encoder based Differentially Private Text Transformation](https://mlsanity.com/paper/2102.01502/) (2021)
- [When differential privacy meets NLP: The devil is in the detail](https://mlsanity.com/paper/2109.03175/) (2021)
- [Privacy-Preserving Graph Convolutional Networks for Text Classification](https://mlsanity.com/paper/2102.09604/) (2021)
- [I Am Not What I Write: Privacy Preserving Text Representation Learning](https://mlsanity.com/paper/1907.03189/) (2019)
- [CheckDP: An Automated and Integrated Approach for Proving Differential Privacy or Finding Precise Counterexamples](https://mlsanity.com/paper/2008.07485/) (2020)

## Built on by (most cited first, 10 of 10)

- [Differentially Private Language Models for Secure Data Sharing](https://mlsanity.com/paper/2210.13918/) (2022)
- [DP-Rewrite: Towards Reproducibility and Transparency in Differentially Private Text Rewriting](https://mlsanity.com/paper/2208.10400/) (2022)
- [DP-BART for Privatized Text Rewriting under Local Differential Privacy](https://mlsanity.com/paper/2302.07636/) (2023)
- [Differential Privacy in Natural Language Processing: The Story So Far](https://mlsanity.com/paper/2208.08140/) (2022)
- [Privacy-Preserving Models for Legal Natural Language Processing](https://mlsanity.com/paper/2211.02956/) (2022)
- [DP-NMT: Scalable Differentially-Private Machine Translation](https://mlsanity.com/paper/2311.14465/) (2023)
- [Trade-Offs Between Fairness and Privacy in Language Modeling](https://mlsanity.com/paper/2305.14936/) (2023)
- [A Neighbourhood-Aware Differential Privacy Mechanism for Static Word Embeddings](https://mlsanity.com/paper/2309.10551/) (2023)
- [Private Language Models via Truncated Laplacian Mechanism](https://mlsanity.com/paper/2410.08027/) (2024)
- [RadProPoser: Probabilistic Radar Tensor Human Pose Estimation That Knows Its Limits](https://mlsanity.com/paper/2508.03578/) (2025)

## Abstract (arXiv)

As privacy gains traction in the NLP community, researchers have started adopting various approaches to privacy-preserving methods. One of the favorite privacy frameworks, differential privacy (DP), is perhaps the most compelling thanks to its fundamental theoretical guarantees. Despite the apparent simplicity of the general concept of differential privacy, it seems non-trivial to get it right when applying it to NLP. In this short paper, we formally analyze several recent NLP papers proposing text representation learning using DPText (Beigi et al., 2019a,b; Alnasser et al., 2021; Beigi et al., 2021) and reveal their false claims of being differentially private. Furthermore, we also show a simple yet general empirical sanity check to determine whether a given implementation of a DP mechanism almost certainly violates the privacy loss guarantees. Our main goal is to raise awareness and help the community understand potential pitfalls of applying differential privacy to text representation learning.