# Automated Profile Inference with Language Model Agents

arXiv: https://arxiv.org/abs/2505.12402 · HTML: https://mlsanity.com/paper/2505.12402/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 22 of its 72 references that are in the map.
- First posted on arXiv 18 May 2025.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 22 of 22)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [A Survey on In-context Learning](https://mlsanity.com/paper/2301.00234/) (2022)
- [Analyzing Leakage of Personally Identifiable Information in Language Models](https://mlsanity.com/paper/2302.00539/) (2023)
- [Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory](https://mlsanity.com/paper/2310.17884/) (2023)
- [Does Prompt Formatting Have Any Impact on LLM Performance?](https://mlsanity.com/paper/2411.10541/) (2024)
- [AgentScope: A Flexible yet Robust Multi-Agent Platform](https://mlsanity.com/paper/2402.14034/) (2024)
- [LLM Agents can Autonomously Hack Websites](https://mlsanity.com/paper/2402.06664/) (2024)
- [LLMs Cannot Reliably Identify and Reason About Security Vulnerabilities (Yet?): A Comprehensive Evaluation, Framework, and Benchmarks](https://mlsanity.com/paper/2312.12575/) (2023)
- [Are Attribute Inference Attacks Just Imputation?](https://mlsanity.com/paper/2209.01292/) (2022)
- [Are Your Sensitive Attributes Private? Novel Model Inversion Attribute Inference Attacks on Classification Models](https://mlsanity.com/paper/2201.09370/) (2022)
- [A Synthetic Dataset for Personal Attribute Inference](https://mlsanity.com/paper/2406.07217/) (2024)
- [A LLM Assisted Exploitation of AI-Guardian](https://mlsanity.com/paper/2307.15008/) (2023)
- [Evaluating LLM-based Personal Information Extraction and Countermeasures](https://mlsanity.com/paper/2408.07291/) (2024)
- [On the Internet, Nobody Knows You're a Dog... Unless You're Another Dog](https://mlsanity.com/paper/2105.03181/) (2021)

## Built on by (most cited first, 9 of 9)

- [A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasures](https://mlsanity.com/paper/2506.19676/) (2025)
- [Large-scale online deanonymization with LLMs](https://mlsanity.com/paper/2602.16800/) (2026)
- [Beyond Data Privacy: New Privacy Risks for Large Language Models](https://mlsanity.com/paper/2509.14278/) (2025)
- [From Weak Cues to Real Identities: Evaluating Inference-Driven De-Anonymization in LLM Agents](https://mlsanity.com/paper/2603.18382/) (2026)
- [What Your Posts Reveal: A Benchmark and Agentic Framework for User-Level Privacy Leakage on Social Media](https://mlsanity.com/paper/2606.06784/) (2026)
- [Profiling for Pennies: Unveiling the Privacy Iceberg of LLM Agents](https://mlsanity.com/paper/2605.06232/) (2026)
- [Inferring Hidden User Models from the Behavior of Personalized LLM Agents](https://mlsanity.com/paper/2609.03815/) (2026)
- [GraphProfiler: Source-Linked Sensitive Attribute Inference via Personal Knowledge Graphs](https://mlsanity.com/paper/2609.12448/) (2026)
- [Privacy Personalization Trade offs in LLMs: The Impact of Stylometric Signal Reduction on User-Specific Text Generation](https://mlsanity.com/paper/2609.22112/) (2026)

## Abstract (arXiv)

Impressive progress has been made in automated problem-solving by the collaboration of large language model (LLM) based agents. However, these automated capabilities also open avenues for malicious applications. In this paper, we study a new threat that LLMs pose to online pseudonymity, called automated profile inference, where an adversary can instruct LLMs to automatically collect and extract sensitive personal attributes from publicly available user activities on pseudonymous platforms. We also introduce an automated profiling framework called AutoProfiler to demonstrate and assess the feasibility of such attacks in real-world scenarios. AutoProfiler consists of four specialized LLM agents that work collaboratively to retrieve and process user online activities and generate a profile with extracted personal information. Experimental results on two real-world datasets and one synthetic dataset show that AutoProfiler is highly effective and efficient, and the inferred attributes are both identifiable and sensitive, posing significant privacy risks. We explore mitigation strategies from different perspectives and advocate for increased public awareness of this emerging privacy threat.