From Weak Cues to Real Identities: Evaluating Inference-Driven De-Anonymization in LLM Agents
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
Anonymization is often assumed to protect privacy once explicit identifiers are removed, because re-identification has historically required specialized expertise, tailored algorithms, and manual corroboration. We show that LLM-based agents weaken this barrier: by combining scattered, individually non-identifying cues with public evidence, they reconstruct real-world identities, sometimes even during benign tasks. We evaluate this risk across three settings -- classical linkage incidents, a controlled benchmark (InferLink) that varies fingerprint type, task framing, and attacker knowledge, and open-ended human--AI interaction traces. In the sparsest regime of the Netflix Prize deanonymization setting, agents reconstruct 79.2% of identities, against 56.0% for a classical matching baseline; on InferLink, they link individuals even without an explicit re-identification request, and more often once one is given. In redacted human--AI interaction traces, agents further resolve anonymized profiles to specific individuals by corroborating contextual cues with public evidence. These findings suggest that privacy evaluations for agentic systems should measure not only what information is accessed or disclosed, but also what identities can be inferred.
Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 17 of its 26 references that are in the map. First posted on arXiv 19 Mar 2026.
Builds on
17 of 26 references.
Closest ancestors · 4
Same direction, newest first
- 2025Automated Profile Inference with Language Model Agents18 May
- 2025WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks22 Apr
- 2025AgentDAM: Privacy Leakage Evaluation for Autonomous Web Agents12 Mar
- 2024AirGapAgent: Protecting Privacy-Conscious Conversational Agents8 May
Other references · 13
- 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
- 2020Extracting Training Data from Large Language Modelscited by 2,189
- 2021Membership Inference Attacks From First Principlescited by 878
- 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theorycited by 230
- 2023Beyond Memorization: Violating Privacy Via Inference with Large Language Modelscited by 175
- 2024PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Actioncited by 171
- 2024The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG)cited by 159
- 2023Practical Membership Inference Attacks Against Large-Scale Multi-Modal Models: A Pilot Studycited by 38
- 2025CIMemories: A Compositional Benchmark for Contextual Integrity of Persistent Memory in LLMscited by 26
- 2026Large-scale online deanonymization with LLMscited by 14
- 2026Agentic LLMs as Powerful Deanonymizers: Re-identification of Participants in the Anthropic Interviewer Datasetcited by 7
- 2025Web-Browsing LLMs Can Access Social Media Profiles and Infer User Demographicscited by 3
1 more
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026ProtoMedAgent: Multimodal Clinical Interpretability via Privacy-Aware Agentic Workflowscited by 2
- 2026LLM Anonymization Against Agentic Re-Identificationcited by 0
- 2026Agentic AI-Powered Re-Identification: An Emerging, Scalable Threat to Mobility Microdata Privacycited by 0
- 2026AgentDoxx: Agentic Re-identification of Anonymized Text with Web Searchcited by 0
- 2026AgentDoxx: Agentic Re-identification of Anonymized Text with Web Search4 Oct 2026
- 2026Agentic AI-Powered Re-Identification: An Emerging, Scalable Threat to Mobility Microdata Privacy26 Jun 2026
- 2026LLM Anonymization Against Agentic Re-Identification29 May 2026
- 2026ProtoMedAgent: Multimodal Clinical Interpretability via Privacy-Aware Agentic Workflows13 May 2026
Similar, not cited
Close in wording, no citation either way
- 2026Privacy-Preserving RAG via Multi-Agent Semantic Rewriting: Achieving Confidentiality Without Compromising Contextual Fidelity
- 2026OCELOT: Inference-Leakage Budgets for Privacy-Preserving LLM Agents
- 2026The Claws in Plain Sight: Unauthorized Context Disclosure through LLM Agent Tool Calls
- 2026Confuse the Model, Control the Flow: Understanding and Mitigating Privacy Leakage from LLM Agents with Information Flow Control
- 2026BodhiPromptShield: Pre-Inference Prompt Mediation for Surface-Form Privacy Propagation in LLM Agent Pipelines
- 2026PrivacyAlign: Contextual Privacy Alignment for LLM Agents
- 2025Exposing LLM User Privacy via Traffic Fingerprint Analysis: A Study of Privacy Risks in LLM Agent Interactions
- 2026PrivacySkills: How Privacy Guidance Shapes Source Selection in LLM Agents