# TextHide: Tackling Data Privacy in Language Understanding Tasks

arXiv: https://arxiv.org/abs/2010.06053 · HTML: https://mlsanity.com/paper/2010.06053/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Oct 2020.

## Built on by (most cited first, 30 of 33)

- [Large Language Models Can Be Strong Differentially Private Learners](https://mlsanity.com/paper/2110.05679/) (2021)
- [The Text Anonymization Benchmark (TAB): A Dedicated Corpus and Evaluation Framework for Text Anonymization](https://mlsanity.com/paper/2202.00443/) (2022)
- [Differential Privacy for Text Analytics via Natural Text Sanitization](https://mlsanity.com/paper/2106.01221/) (2021)
- [Recovering Private Text in Federated Learning of Language Models](https://mlsanity.com/paper/2205.08514/) (2022)
- [THE-X: Privacy-Preserving Transformer Inference with Homomorphic Encryption](https://mlsanity.com/paper/2206.00216/) (2022)
- [Privacy-Preserving Parameter-Efficient Fine-Tuning for Large Language Model Services](https://mlsanity.com/paper/2305.06212/) (2023)
- [Backdoor Attacks and Countermeasures in Natural Language Processing Models: A Comprehensive Security Review](https://mlsanity.com/paper/2309.06055/) (2023)
- [CaPC Learning: Confidential and Private Collaborative Learning](https://mlsanity.com/paper/2102.05188/) (2021)
- [Privacy Implications of Retrieval-Based Language Models](https://mlsanity.com/paper/2305.14888/) (2023)
- [Split-and-Denoise: Protect large language model inference with local differential privacy](https://mlsanity.com/paper/2310.09130/) (2023)
- [Text Embedding Inversion Security for Multilingual Language Models](https://mlsanity.com/paper/2401.12192/) (2024)
- [Sentence-level Privacy for Document Embeddings](https://mlsanity.com/paper/2205.04605/) (2022)
- [How to keep text private? A systematic review of deep learning methods for privacy-preserving natural language processing](https://mlsanity.com/paper/2205.10095/) (2022)
- [On Convergence of Federated Averaging Langevin Dynamics](https://mlsanity.com/paper/2112.05120/) (2021)
- [The Fire Thief Is Also the Keeper: Balancing Usability and Privacy in Prompts](https://mlsanity.com/paper/2406.14318/) (2024)
- [Federated Split BERT for Heterogeneous Text Classification](https://mlsanity.com/paper/2205.13299/) (2022)
- [Dropout is NOT All You Need to Prevent Gradient Leakage](https://mlsanity.com/paper/2208.06163/) (2022)
- [Federated Empirical Risk Minimization via Second-Order Method](https://mlsanity.com/paper/2305.17482/) (2023)
- [A Method to Reveal Speaker Identity in Distributed ASR Training, and How to Counter It](https://mlsanity.com/paper/2104.07815/) (2021)
- [FedVLN: Privacy-preserving Federated Vision-and-Language Navigation](https://mlsanity.com/paper/2203.14936/) (2022)
- [Practical and Secure Federated Recommendation with Personalized Masks](https://mlsanity.com/paper/2109.02464/) (2021)
- [Fast John Ellipsoid Computation with Differential Privacy Optimization](https://mlsanity.com/paper/2408.06395/) (2024)
- [Efficiency 360: Efficient Vision Transformers](https://mlsanity.com/paper/2302.08374/) (2023)
- [Too Brittle To Touch: Comparing the Stability of Quantization and Distillation Towards Developing Lightweight Low-Resource MT Models](https://mlsanity.com/paper/2210.15184/) (2022)
- [Dropout against Deep Leakage from Gradients](https://mlsanity.com/paper/2108.11106/) (2021)
- [PPFedIT: Towards Privacy-Preserving Federated Instruction Tuning with Few-shot Local Examples](https://mlsanity.com/paper/2403.06131/) (2024)
- [Privacy Evaluation Benchmarks for NLP Models](https://mlsanity.com/paper/2409.15868/) (2024)
- [Combining Stochastic Defenses to Resist Gradient Inversion: An Ablation Study](https://mlsanity.com/paper/2208.04767/) (2022)
- [LatticeGen: A Cooperative Framework which Hides Generated Text in a Lattice for Privacy-Aware Generation on Cloud](https://mlsanity.com/paper/2309.17157/) (2023)
- [NLP Security and Ethics, in the Wild](https://mlsanity.com/paper/2504.06669/) (2025)

## Abstract (arXiv)

An unsolved challenge in distributed or federated learning is to effectively mitigate privacy risks without slowing down training or reducing accuracy. In this paper, we propose TextHide aiming at addressing this challenge for natural language understanding tasks. It requires all participants to add a simple encryption step to prevent an eavesdropping attacker from recovering private text data. Such an encryption step is efficient and only affects the task performance slightly. In addition, TextHide fits well with the popular framework of fine-tuning pre-trained language models (e.g., BERT) for any sentence or sentence-pair task. We evaluate TextHide on the GLUE benchmark, and our experiments show that TextHide can effectively defend attacks on shared gradients or representations and the averaged accuracy reduction is only 1.9%. We also present an analysis of the security of TextHide using a conjecture about the computational intractability of a mathematical problem. Our code is available at https://github.com/Hazelsuko07/TextHide