A Longitudinal Measurement of Privacy Policy Evolution for Large Language Models
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
Large language model (LLM) services have been rapidly integrated into people's daily lives as chatbots and agentic systems. They are nourished by collecting rich streams of data, raising privacy concerns around excessive collection of sensitive personal information. Privacy policies are the fundamental mechanism for informing users about data practices in modern information privacy paradigm. Although traditional web and mobile policies are well studied, the privacy policies of LLM providers, their LLM-specific content, and their evolution over time remain largely underexplored. In this paper, we present the first longitudinal empirical study of privacy policies for mainstream LLM providers worldwide. We curate a chronological dataset of 74 historical privacy policies and 115 supplemental privacy documents from 11 LLM providers across 5 countries up to August 2025, and extract over 3,000 sentence-level edits between consecutive policy versions. We compare LLM privacy policies to those of other software formats, propose a taxonomy tailored to LLM privacy policies, annotate policy edits and align them with a timeline of key LLM ecosystem events. Results show they are substantially longer, demand college-level reading ability, and remain highly vague. Our taxonomy analysis reveals patterns in how providers disclose LLM-specific practices and highlights regional disparities in coverage. Policy edits are concentrated in first-party data collection and international/specific-audience sections, and that product releases and regulatory actions are the primary drivers, shedding light on the status quo and the evolution of LLM privacy policies.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 48 of its 128 references that are in the map. First posted on arXiv 24 Nov 2025.
Builds on
48 of 128 references.
Closest ancestors · 3
Same direction, newest first
- 2025Unveiling Privacy Risks in LLM Agent Memory17 Feb
- 2024Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agents14 Nov
- 2024The Emerged Security and Privacy of LLM Agent: A Survey with Case Studies28 Jul
Other references · 45
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,184
- 2021Membership Inference Attacks From First Principlescited by 878
- 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 560
- 2022Are Large Pre-Trained Language Models Leaking Your Personal Information?cited by 200
- 2022Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attackscited by 175
- 2023From ChatGPT to ThreatGPT: Impact of Generative AI in Cybersecurity and Privacycited by 143
- 2024Multi-IF: Benchmarking LLMs on Multi-Turn and Multilingual Instructions Followingcited by 102
- 2024AIR-Bench 2024: A Safety Benchmark Based on Risk Categories from Regulations and Policiescited by 74
- 2023Privacy Issues in Large Language Models: A Surveycited by 60
33 more
- 2023Did the Neurons Read your Book? Document-level Membership Inference for Large Language Modelscited by 56
- 2023Unveiling Security, Privacy, and Ethical Concerns of ChatGPTcited by 53
- 2023LLM Platform Security: Applying a Systematic Evaluation Framework to OpenAI's ChatGPT Pluginscited by 51
- 2023Privacy in Large Language Models: Attacks, Defenses and Future Directionscited by 51
- 2018Polisis: Automated Analysis and Presentation of Privacy Policies Using Deep Learningcited by 50
- 2025MCP-RADAR: A Multi-Dimensional Benchmark for Evaluating Tool Use Capabilities in Large Language Modelscited by 48
- 2023The Philosopher's Stone: Trojaning Plugins of Large Language Modelscited by 37
- 2020PolicyQA: A Reading Comprehension Dataset for Privacy Policiescited by 34
- 2024CLEAR: Towards Contextual LLM-Empowered Privacy Policy Analysis and Risk Generation for Large Language Model Applicationscited by 33
- 2025Membership Inference Attacks Against Vision-Language Modelscited by 29
- 2020Privacy Policies over Time: Curation and Analysis of a Million-Document Datasetcited by 28
- 2020Privacy at Scale: Introducing the PrivaSeer Corpus of Web Privacy Policiescited by 22
- 2018The Privacy Policy Landscape After the GDPRcited by 18
- 2018We Value Your Privacy ... Now Take Some Cookies: Measuring the GDPR's Impact on Web Privacycited by 15
- 2024An In-Depth Investigation of Data Collection in LLM App Ecosystemscited by 14
- 2025Exploring User Security and Privacy Attitudes and Concerns Toward the Use of General-Purpose LLM Chatbots for Mental Healthcited by 13
- 2022PoliGraph: Automated Privacy Policy Analysis using Knowledge Graphs (Journal Version)cited by 10
- 2024"Ghost of the past": identifying and resolving privacy leakage from LLM's memory through proactive user interactioncited by 10
- 2025Understanding Users' Security and Privacy Concerns and Attitudes Towards Conversational AI Platformscited by 9
- 2025RealFactBench: A Benchmark for Evaluating Large Language Models in Real-World Fact-Checkingcited by 9
- 2025User Privacy and Large Language Models: An Analysis of Frontier Developers' Privacy Policiescited by 9
- 2018An Automated Approach to Auditing Disclosure of Third-Party Data Collection in Website Privacy Policiescited by 5
- 2018Modeling Language Vagueness in Privacy Policies using Deep Neural Networkscited by 5
- 2023Is It a Trap? A Large-scale Empirical Study And Comprehensive Assessment of Online Automated Privacy Policy Generators for Mobile Appscited by 5
- 2025Privacy Bills of Materials: A Transparent Privacy Information Inventory for Collaborative Privacy Notice Generation in Mobile App Developmentcited by 5
- 2020Quality Assessment of Online Automated Privacy Policy Generators: An Empirical Studycited by 4
- 2024{A New Hope}: Contextual Privacy Policies for Mobile Applications and An Approach Toward Automated Generationcited by 4
- 2025Big Help or Big Brother? Auditing Tracking, Profiling, and Personalization in Generative AI Assistantscited by 4
- 2020On the (Un)Reliability of Privacy Policies in Android Appscited by 2
- 2020Calpric: Inclusive and Fine-grain Labeling of Privacy Policies with Crowdsourcing and Active Learningcited by 2
- 2023More Data Types More Problems: A Temporal Analysis of Complexity, Stability, and Sensitivity in Privacy Policiescited by 1
- 2025Overcoming Language Barriers: Multilingual Analysis of the 2023 Swiss Privacy Law's Impactcited by 1
- 2025A Big Step Forward? A User-Centric Examination of iOS App Privacy Report and Enhancementscited by 1
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025SoK: The Privacy Paradox of Large Language Models: Advancements, Privacy Risks, and Mitigation
- 2026Privacy Control in Conversational LLM Platforms: A Walkthrough Study
- 2025The Model's Language Matters: A Comparative Privacy Analysis of LLMs
- 2025User Behavior Analysis in Privacy Protection with Large Language Models: A Study on Privacy Preferences with Limited Data
- 2024Empowering Users in Digital Privacy Management through Interactive LLM-Based Agents
- 2025Controlling What You Share: Assessing Language Model Adherence to Privacy Preferences
- 2025A Survey on Privacy Risks and Protection in Large Language Models
- 2025Deploying Privacy Guardrails for LLMs: A Comparative Analysis of Real-World Applications