# Large Language Models are Advanced Anonymizers

arXiv: https://arxiv.org/abs/2402.13846 · HTML: https://mlsanity.com/paper/2402.13846/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 30 references that are in the map.
- First posted on arXiv 21 Feb 2024.

## Builds on (most cited first, 13 of 13)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Quantifying Memorization Across Neural Language Models](https://mlsanity.com/paper/2202.07646/) (2022)
- [Can Large Language Models Be an Alternative to Human Evaluations?](https://mlsanity.com/paper/2305.01937/) (2023)
- [Beyond Memorization: Violating Privacy Via Inference with Large Language Models](https://mlsanity.com/paper/2310.07298/) (2023)
- [The Text Anonymization Benchmark (TAB): A Dedicated Corpus and Evaluation Framework for Text Anonymization](https://mlsanity.com/paper/2202.00443/) (2022)
- [De-identification of Patient Notes with Recurrent Neural Networks](https://mlsanity.com/paper/1606.03475/) (2016)
- [The Limitations of Stylometry for Detecting Machine-Generated Fake News](https://mlsanity.com/paper/1908.09805/) (2019)
- [Reducing Privacy Risks in Online Self-Disclosures with Language Models](https://mlsanity.com/paper/2311.09538/) (2023)
- [C-sanitized: a privacy model for document redaction and sanitization](https://mlsanity.com/paper/1406.4285/) (2014)

## Built on by (most cited first, 22 of 22)

- [Private Attribute Inference from Images with Vision-Language Models](https://mlsanity.com/paper/2404.10618/) (2024)
- [A Synthetic Dataset for Personal Attribute Inference](https://mlsanity.com/paper/2406.07217/) (2024)
- [PrivacyRestore: Privacy-Preserving Inference in Large Language Models via Privacy Removal and Restoration](https://mlsanity.com/paper/2406.01394/) (2024)
- [IncogniText: Privacy-enhancing Conditional Text Anonymization via LLM-based Private Attribute Randomization](https://mlsanity.com/paper/2407.02956/) (2024)
- [The Sum Leaks More Than Its Parts: Compositional Privacy Risks and Mitigations in Multi-Agent Collaboration](https://mlsanity.com/paper/2509.14284/) (2025)
- [A False Sense of Privacy: Evaluating Textual Data Sanitization Beyond Surface-level Privacy Leakage](https://mlsanity.com/paper/2504.21035/) (2025)
- [Large-scale online deanonymization with LLMs](https://mlsanity.com/paper/2602.16800/) (2026)
- [PBa-LLM: Privacy- and Bias-aware NLP using Named-Entity Recognition (NER)](https://mlsanity.com/paper/2507.02966/) (2025)
- [NAP^2: A Benchmark for Naturalness and Privacy-Preserving Text Rewriting by Learning from Human](https://mlsanity.com/paper/2406.03749/) (2024)
- [Information-Theoretic Privacy Control for Sequential Multi-Agent LLM Systems](https://mlsanity.com/paper/2603.05520/) (2026)
- [Unlocking the Potential of Large Language Models for Clinical Text Anonymization: A Comparative Study](https://mlsanity.com/paper/2406.00062/) (2024)
- [PPMI: Privacy-Preserving LLM Interaction with Socratic Chain-of-Thought Reasoning and Homomorphically Encrypted Vector Databases](https://mlsanity.com/paper/2506.17336/) (2025)
- [DP-Fusion: Token-Level Differentially Private Inference for Large Language Models](https://mlsanity.com/paper/2507.04531/) (2025)
- [RewardDS: Privacy-Preserving Fine-Tuning for Large Language Models via Reward Driven Data Synthesis](https://mlsanity.com/paper/2502.18517/) (2025)
- [Zero-Shot Privacy-Aware Text Rewriting via Iterative Tree Search](https://mlsanity.com/paper/2509.20838/) (2025)
- [Retrieval-Confused Generation is a Good Defender for Privacy Violation Attack of Large Language Models](https://mlsanity.com/paper/2506.19889/) (2025)
- [Controlling What You Share: Assessing Language Model Adherence to Privacy Preferences](https://mlsanity.com/paper/2507.05391/) (2025)
- [Improving LLM's Attachment to External Knowledge In Dialogue Generation Tasks Through Entity Anonymization](https://mlsanity.com/paper/2511.11946/) (2025)
- [A Queueing-Theoretic Framework for Dynamic Attack Surfaces: Data-Integrated Risk Analysis and Adaptive Defense](https://mlsanity.com/paper/2604.10427/) (2026)
- [PromptPET: Privacy-Utility Optimized Prompt Obfuscation](https://mlsanity.com/paper/2607.02932/) (2026)
- [LLMs as Linguistic Chameleons: Decoupling Semantics and Structure for Privacy-Preserving Communication](https://mlsanity.com/paper/2609.23193/) (2026)
- [Reading Is Not Leaking: Local, Auditable Measurement and Reduction of Inference Exposure from Public Footprints](https://mlsanity.com/paper/2609.32565/) (2026)

## Abstract (arXiv)

Recent privacy research on large language models (LLMs) has shown that they achieve near-human-level performance at inferring personal data from online texts. With ever-increasing model capabilities, existing text anonymization methods are currently lacking behind regulatory requirements and adversarial threats. In this work, we take two steps to bridge this gap: First, we present a new setting for evaluating anonymization in the face of adversarial LLM inferences, allowing for a natural measurement of anonymization performance while remedying some of the shortcomings of previous metrics. Then, within this setting, we develop a novel LLM-based adversarial anonymization framework leveraging the strong inferential capabilities of LLMs to inform our anonymization procedure. We conduct a comprehensive experimental evaluation of adversarial anonymization across 13 LLMs on real-world and synthetic online texts, comparing it against multiple baselines and industry-grade anonymizers. Our evaluation shows that adversarial anonymization outperforms current commercial anonymizers both in terms of the resulting utility and privacy. We support our findings with a human study (n=50) highlighting a strong and consistent human preference for LLM-anonymized texts.