# IncogniText: Privacy-enhancing Conditional Text Anonymization via LLM-based Private Attribute Randomization

arXiv: https://arxiv.org/abs/2407.02956 · HTML: https://mlsanity.com/paper/2407.02956/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 35 references that are in the map.
- First posted on arXiv 3 Jul 2024.

## Builds on (most cited first, 14 of 14)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [Beyond Memorization: Violating Privacy Via Inference with Large Language Models](https://mlsanity.com/paper/2310.07298/) (2023)
- [AutoAttacker: A Large Language Model Guided System to Implement Automatic Cyber-attacks](https://mlsanity.com/paper/2403.01038/) (2024)
- [The Text Anonymization Benchmark (TAB): A Dedicated Corpus and Evaluation Framework for Text Anonymization](https://mlsanity.com/paper/2202.00443/) (2022)
- [Privacy Issues in Large Language Models: A Survey](https://mlsanity.com/paper/2312.06717/) (2023)
- [A Short Survey of Viewing Large Language Models in Legal Aspect](https://mlsanity.com/paper/2303.09136/) (2023)
- [Reducing Privacy Risks in Online Self-Disclosures with Language Models](https://mlsanity.com/paper/2311.09538/) (2023)
- [Privacy-Preserving Parameter-Efficient Fine-Tuning for Large Language Model Services](https://mlsanity.com/paper/2305.06212/) (2023)
- [DP-BART for Privatized Text Rewriting under Local Differential Privacy](https://mlsanity.com/paper/2302.07636/) (2023)
- [Large Language Models are Advanced Anonymizers](https://mlsanity.com/paper/2402.13846/) (2024)
- [Text Sanitization Beyond Specific Domains: Zero-Shot Redaction & Substitution with Large Language Models](https://mlsanity.com/paper/2311.10785/) (2023)

## Built on by (most cited first, 23 of 23)

- [A False Sense of Privacy: Evaluating Textual Data Sanitization Beyond Surface-level Privacy Leakage](https://mlsanity.com/paper/2504.21035/) (2025)
- [Robust Utility-Preserving Text Anonymization Based on Large Language Models](https://mlsanity.com/paper/2407.11770/) (2024)
- [A Survey on Current Trends and Recent Advances in Text Anonymization](https://mlsanity.com/paper/2508.21587/) (2025)
- [How do we measure privacy in text? A survey of text anonymization metrics](https://mlsanity.com/paper/2512.01109/) (2025)
- [Can Large Language Models Really Recognize Your Name?](https://mlsanity.com/paper/2505.14549/) (2025)
- [Self-Refining Language Model Anonymizers via Adversarial Distillation](https://mlsanity.com/paper/2506.01420/) (2025)
- [Stop Tracking Me! Proactive Defense Against Attribute Inference Attack in LLMs](https://mlsanity.com/paper/2602.11528/) (2026)
- [Learning to Erase Private Knowledge from Multi-Documents for Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2504.09910/) (2025)
- [Private Seeds, Public LLMs: Realistic and Privacy-Preserving Synthetic Data Generation](https://mlsanity.com/paper/2604.07486/) (2026)
- [Towards Privacy-Preserving and Personalized Smart Homes via Tailored Small Language Models](https://mlsanity.com/paper/2507.08878/) (2025)
- [A Survey: Towards Privacy and Security in Mobile Large Language Models](https://mlsanity.com/paper/2509.02411/) (2025)
- [Adaptive Text Anonymization: Learning Privacy-Utility Trade-offs via Prompt Optimization](https://mlsanity.com/paper/2602.20743/) (2026)
- [Subject-level Inference for Realistic Text Anonymization Evaluation](https://mlsanity.com/paper/2604.21211/) (2026)
- [GRASP: Reinforcing Language Model Anonymizers with Group Relative Policy Optimization](https://mlsanity.com/paper/2608.06526/) (2026)
- [Retrieval-Confused Generation is a Good Defender for Privacy Violation Attack of Large Language Models](https://mlsanity.com/paper/2506.19889/) (2025)
- [Protecting De-identified Documents from Search-based Linkage Attacks](https://mlsanity.com/paper/2510.06383/) (2025)
- [From Fragmentation to Integration: Exploring the Design Space of AI Agents for Human-as-the-Unit Privacy Management](https://mlsanity.com/paper/2602.05016/) (2026)
- [Introducing the Privacy-HSD Trade-off: Hate Speech Detection, but not at the Cost of Privacy](https://mlsanity.com/paper/2608.19006/) (2026)
- [GuardianAgent: Policy-Conditioned Risk-Adaptive Anonymization with Verified Adversarial Escalation](https://mlsanity.com/paper/2608.29251/) (2026)
- [PrivBench: A Holistic and Modular Benchmarking Platform for Evaluating Text-to-Text Privatization](https://mlsanity.com/paper/2608.29624/) (2026)
- [GraphProfiler: Source-Linked Sensitive Attribute Inference via Personal Knowledge Graphs](https://mlsanity.com/paper/2609.12448/) (2026)
- [LLMs as Linguistic Chameleons: Decoupling Semantics and Structure for Privacy-Preserving Communication](https://mlsanity.com/paper/2609.23193/) (2026)
- [AgentDoxx: Agentic Re-identification of Anonymized Text with Web Search](https://mlsanity.com/paper/2610.05586/) (2026)

## Abstract (arXiv)

In this work, we address the problem of text anonymization where the goal is to prevent adversaries from correctly inferring private attributes of the author, while keeping the text utility, i.e., meaning and semantics. We propose IncogniText, a technique that anonymizes the text to mislead a potential adversary into predicting a wrong private attribute value. Our empirical evaluation shows a reduction of private attribute leakage by more than 90% across 8 different private attributes. Finally, we demonstrate the maturity of IncogniText for real-world applications by distilling its anonymization capability into a set of LoRA parameters associated with an on-device model. Our results show the possibility of reducing privacy leakage by more than half with limited impact on utility.