# Profiling for Pennies: Unveiling the Privacy Iceberg of LLM Agents

arXiv: https://arxiv.org/abs/2605.06232 · HTML: https://mlsanity.com/paper/2605.06232/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 19 of its 35 references that are in the map.
- First posted on arXiv 7 May 2026.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 19 of 19)

- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Formalizing and Benchmarking Prompt Injection Attacks and Defenses](https://mlsanity.com/paper/2310.12815/) (2023)
- [Large Language Models for Data Annotation and Synthesis: A Survey](https://mlsanity.com/paper/2402.13446/) (2024)
- [Beyond Memorization: Violating Privacy Via Inference with Large Language Models](https://mlsanity.com/paper/2310.07298/) (2023)
- [Graph Unlearning](https://mlsanity.com/paper/2103.14991/) (2021)
- [xbench: Tracking Agents Productivity Scaling with Profession-Aligned Real-World Evaluations](https://mlsanity.com/paper/2506.13651/) (2025)
- [Exploring LLM-based Agents for Root Cause Analysis](https://mlsanity.com/paper/2403.04123/) (2024)
- [Private Attribute Inference from Images with Vision-Language Models](https://mlsanity.com/paper/2404.10618/) (2024)
- [Comparing Traditional and LLM-based Search for Consumer Choice: A Randomized Experiment](https://mlsanity.com/paper/2307.03744/) (2023)
- [The Janus Interface: How Fine-Tuning in Large Language Models Amplifies the Privacy Risks](https://mlsanity.com/paper/2310.15469/) (2023)
- [A Synthetic Dataset for Personal Attribute Inference](https://mlsanity.com/paper/2406.07217/) (2024)
- [When LLMs Go Online: The Emerging Threat of Web-Enabled LLMs](https://mlsanity.com/paper/2410.14569/) (2024)
- [Evaluating LLM-based Personal Information Extraction and Countermeasures](https://mlsanity.com/paper/2408.07291/) (2024)
- [Ethical Frameworks and Computer Security Trolley Problems: Foundations for Conversations](https://mlsanity.com/paper/2302.14326/) (2023)
- [Automated Profile Inference with Language Model Agents](https://mlsanity.com/paper/2505.12402/) (2025)
- [Leveraging Large Language Models to Enhance Personalized Recommendations in E-commerce](https://mlsanity.com/paper/2410.12829/) (2024)
- [Scrub It Out! Erasing Sensitive Memorization in Code Language Models via Machine Unlearning](https://mlsanity.com/paper/2509.13755/) (2025)
- [Beyond Data Privacy: New Privacy Risks for Large Language Models](https://mlsanity.com/paper/2509.14278/) (2025)
- [On the Privacy Risks of Cell-Based NAS Architectures](https://mlsanity.com/paper/2209.01688/) (2022)

## Abstract (arXiv)

Large Language Models (LLMs) have revolutionized how information are collected, aggregated, and reasoned. However, this enables a novel and accessible vector of privacy intrusion: the automated and in-depth personal profiling; this engenders a chilling effect of "peepers everywhere". Existing research primarily unfolds from the training pipeline of LLM, emphasizing the exposure of Personally Identifiable Information (PII) through memorization, while privacy studies from a human-centric perspective remain underexplored. To fill this void, we empirically investigate privacy perception in the real world through the lens of human awareness and the practices of LLM-integrated platforms, revealing a significant dissonance: platforms fail to technically or policy-wise address public privacy concerns. To facilitate a systematic and quantifiable study of privacy risk, we propose the PrivacyIceberg, which categorizes real-world human privacy risks into three tiers: explicitly searched, contextually inferred, and deeply aggregated, based on the sophistication of LLM exploitation. We developed IcebergExplorer to audit privacy exposure, utilizing minimal PII as a search seed to reconstruct high-fidelity profiles, achieving over 90% factual accuracy within 10 minutes at a cost under 3, for real-world scenarios. Additionally, we identify six root causes contributing to such privacy disclosures and propose multi-stakeholder countermeasures for LLM vendors, individuals, and data publishers.