# Leveraging Hierarchical Representations for Preserving Privacy and Utility in Text

arXiv: https://arxiv.org/abs/1910.08917 · HTML: https://mlsanity.com/paper/1910.08917/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Oct 2019.

## Built on by (most cited first, 30 of 50)

- [Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipe](https://mlsanity.com/paper/2210.14348/) (2022)
- [The Text Anonymization Benchmark (TAB): A Dedicated Corpus and Evaluation Framework for Text Anonymization](https://mlsanity.com/paper/2202.00443/) (2022)
- [Differential Privacy for Text Analytics via Natural Text Sanitization](https://mlsanity.com/paper/2106.01221/) (2021)
- [Differentially Private Language Models for Secure Data Sharing](https://mlsanity.com/paper/2210.13918/) (2022)
- [DP-Forward: Fine-tuning and Inference on Language Models with Differential Privacy in Forward Pass](https://mlsanity.com/paper/2309.06746/) (2023)
- [The Limits of Word Level Differential Privacy](https://mlsanity.com/paper/2205.02130/) (2022)
- [A Customized Text Sanitization Mechanism with Differential Privacy](https://mlsanity.com/paper/2207.01193/) (2022)
- [ADePT: Auto-encoder based Differentially Private Text Transformation](https://mlsanity.com/paper/2102.01502/) (2021)
- [DP-BART for Privatized Text Rewriting under Local Differential Privacy](https://mlsanity.com/paper/2302.07636/) (2023)
- [PrivacyRestore: Privacy-Preserving Inference in Large Language Models via Privacy Removal and Restoration](https://mlsanity.com/paper/2406.01394/) (2024)
- [Differentially Private Natural Language Models: Recent Advances and Future Directions](https://mlsanity.com/paper/2301.09112/) (2023)
- [Position: Beyond Euclidean -- Foundation Models Should Embrace Non-Euclidean Geometries](https://mlsanity.com/paper/2504.08896/) (2025)
- [On a Utilitarian Approach to Privacy Preserving Text Generation](https://mlsanity.com/paper/2104.11838/) (2021)
- [How to keep text private? A systematic review of deep learning methods for privacy-preserving natural language processing](https://mlsanity.com/paper/2205.10095/) (2022)
- [Guiding Text-to-Text Privatization by Syntax](https://mlsanity.com/paper/2306.01471/) (2023)
- [Privacy in Fine-tuning Large Language Models: Attacks, Defenses, and Future Directions](https://mlsanity.com/paper/2412.16504/) (2024)
- [Driving Context into Text-to-Text Privatization](https://mlsanity.com/paper/2306.01457/) (2023)
- [A Comparative Analysis of Word-Level Metric Differential Privacy: Benchmarking The Privacy-Utility Trade-off](https://mlsanity.com/paper/2404.03324/) (2024)
- [The Fire Thief Is Also the Keeper: Balancing Usability and Privacy in Prompts](https://mlsanity.com/paper/2406.14318/) (2024)
- [You Are What You Write: Preserving Privacy in the Era of Large Language Models](https://mlsanity.com/paper/2204.09391/) (2022)
- [Differential Privacy in Natural Language Processing: The Story So Far](https://mlsanity.com/paper/2208.08140/) (2022)
- [Recent Advances of Differential Privacy in Centralized Deep Learning: A Systematic Survey](https://mlsanity.com/paper/2309.16398/) (2023)
- [The Impact of Differential Privacy on Group Disparity Mitigation](https://mlsanity.com/paper/2203.02745/) (2022)
- [1-Diffractor: Efficient and Utility-Preserving Text Obfuscation Leveraging Word-Level Metric Differential Privacy](https://mlsanity.com/paper/2405.01678/) (2024)
- [Robust Utility-Preserving Text Anonymization Based on Large Language Models](https://mlsanity.com/paper/2407.11770/) (2024)
- [Bootstrapping Text Anonymization Models with Distant Supervision](https://mlsanity.com/paper/2205.06895/) (2022)
- [Cross-Network Social User Embedding with Hybrid Differential Privacy Guarantees](https://mlsanity.com/paper/2209.01539/) (2022)
- [Neural Text Sanitization with Privacy Risk Indicators: An Empirical Analysis](https://mlsanity.com/paper/2310.14312/) (2023)
- [TAROT: Task-Oriented Authorship Obfuscation Using Policy Optimization Methods](https://mlsanity.com/paper/2407.21630/) (2024)
- [Reconstruction of Differentially Private Text Sanitization via Large Language Models](https://mlsanity.com/paper/2410.12443/) (2024)

## Abstract (arXiv)

Guaranteeing a certain level of user privacy in an arbitrary piece of text is a challenging issue. However, with this challenge comes the potential of unlocking access to vast data stores for training machine learning models and supporting data driven decisions. We address this problem through the lens of dx-privacy, a generalization of Differential Privacy to non Hamming distance metrics. In this work, we explore word representations in Hyperbolic space as a means of preserving privacy in text. We provide a proof satisfying dx-privacy, then we define a probability distribution in Hyperbolic space and describe a way to sample from it in high dimensions. Privacy is provided by perturbing vector representations of words in high dimensional Hyperbolic space to obtain a semantic generalization. We conduct a series of experiments to demonstrate the tradeoff between privacy and utility. Our privacy experiments illustrate protections against an authorship attribution algorithm while our utility experiments highlight the minimal impact of our perturbations on several downstream machine learning models. Compared to the Euclidean baseline, we observe > 20x greater guarantees on expected privacy against comparable worst case statistics.