# Towards Differentially Private Text Representations

arXiv: https://arxiv.org/abs/2006.14170 · HTML: https://mlsanity.com/paper/2006.14170/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jun 2020.

## Built on by (most cited first, 29 of 29)

- [Differential Privacy for Text Analytics via Natural Text Sanitization](https://mlsanity.com/paper/2106.01221/) (2021)
- [DP-Forward: Fine-tuning and Inference on Language Models with Differential Privacy in Forward Pass](https://mlsanity.com/paper/2309.06746/) (2023)
- [Natural Language Understanding with Privacy-Preserving BERT](https://mlsanity.com/paper/2104.07504/) (2021)
- [Model Extraction and Adversarial Transferability, Your BERT is Vulnerable!](https://mlsanity.com/paper/2103.10013/) (2021)
- [Privacy Preserving Prompt Engineering: A Survey](https://mlsanity.com/paper/2404.06001/) (2024)
- [FairVFL: A Fair Vertical Federated Learning Framework with Contrastive Adversarial Learning](https://mlsanity.com/paper/2206.03200/) (2022)
- [Differentially Private Natural Language Models: Recent Advances and Future Directions](https://mlsanity.com/paper/2301.09112/) (2023)
- [Privacy-Preserving Graph Convolutional Networks for Text Classification](https://mlsanity.com/paper/2102.09604/) (2021)
- [Active Membership Inference Attack under Local Differential Privacy in Federated Learning](https://mlsanity.com/paper/2302.12685/) (2023)
- [A Comparative Analysis of Word-Level Metric Differential Privacy: Benchmarking The Privacy-Utility Trade-off](https://mlsanity.com/paper/2404.03324/) (2024)
- [The Fire Thief Is Also the Keeper: Balancing Usability and Privacy in Prompts](https://mlsanity.com/paper/2406.14318/) (2024)
- [Recent Advances of Differential Privacy in Centralized Deep Learning: A Systematic Survey](https://mlsanity.com/paper/2309.16398/) (2023)
- [XRand: Differentially Private Defense against Explanation-Guided Attacks](https://mlsanity.com/paper/2212.04454/) (2022)
- [A Split-and-Privatize Framework for Large Language Model Fine-Tuning](https://mlsanity.com/paper/2312.15603/) (2023)
- [EmojiPrompt: Generative Prompt Obfuscation for Privacy-Preserving Communication with Cloud-based LLMs](https://mlsanity.com/paper/2402.05868/) (2024)
- [FLSys: Toward an Open Ecosystem for Federated Learning Mobile Apps](https://mlsanity.com/paper/2111.09445/) (2021)
- [Cross-Network Social User Embedding with Hybrid Differential Privacy Guarantees](https://mlsanity.com/paper/2209.01539/) (2022)
- [SplitGNN: Splitting GNN for Node Classification with Heterogeneous Attention](https://mlsanity.com/paper/2301.12885/) (2023)
- [A Neighbourhood-Aware Differential Privacy Mechanism for Static Word Embeddings](https://mlsanity.com/paper/2309.10551/) (2023)
- [Towards Harnessing the Collaborative Power of Large and Small Models for Domain Tasks](https://mlsanity.com/paper/2504.17421/) (2025)
- [Private Language Models via Truncated Laplacian Mechanism](https://mlsanity.com/paper/2410.08027/) (2024)
- [Differentially Private Learning Needs Better Model Initialization and Self-Distillation](https://mlsanity.com/paper/2410.17566/) (2024)
- [PrIeD-KIE: Towards Privacy Preserved Document Key Information Extraction](https://mlsanity.com/paper/2310.03777/) (2023)
- [Concept-Aware Privacy Mechanisms for Defending Embedding Inversion Attacks](https://mlsanity.com/paper/2602.07090/) (2026)
- [User-Entity Differential Privacy in Learning Natural Language Models](https://mlsanity.com/paper/2211.01141/) (2022)
- [Investigating User Perspectives on Differentially Private Text Privatization](https://mlsanity.com/paper/2503.09338/) (2025)
- [NOIR: Privacy-Preserving Generation of Code with Open-Source LLMs](https://mlsanity.com/paper/2601.16354/) (2026)
- [SharedRequest: Privacy-Preserving Model-Agnostic Inference for Large Language Models](https://mlsanity.com/paper/2606.05004/) (2026)
- [Theoretically Unmasking Inference Attacks Against LDP-Protected Clients in Federated Vision Models](https://mlsanity.com/paper/2506.17292/) (2025)

## Abstract (arXiv)

Most deep learning frameworks require users to pool their local data or model updates to a trusted server to train or maintain a global model. The assumption of a trusted server who has access to user information is ill-suited in many applications. To tackle this problem, we develop a new deep learning framework under an untrusted server setting, which includes three modules: (1) embedding module, (2) randomization module, and (3) classifier module. For the randomization module, we propose a novel local differentially private (LDP) protocol to reduce the impact of privacy parameter ε on accuracy, and provide enhanced flexibility in choosing randomization probabilities for LDP. Analysis and experiments show that our framework delivers comparable or even better performance than the non-private framework and existing LDP protocols, demonstrating the advantages of our LDP protocol.