# PolicyGPT: Automated Analysis of Privacy Policies with Large Language Models

arXiv: https://arxiv.org/abs/2309.10238 · HTML: https://mlsanity.com/paper/2309.10238/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 31 references that are in the map.
- First posted on arXiv 19 Sep 2023.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Visual Programming: Compositional visual reasoning without training](https://mlsanity.com/paper/2211.11559/) (2022)
- [DeID-GPT: Zero-shot Medical Text De-Identification by GPT-4](https://mlsanity.com/paper/2303.11032/) (2023)
- [An Iterative Optimizing Framework for Radiology Report Summarization with ChatGPT](https://mlsanity.com/paper/2304.08448/) (2023)
- [When Brain-inspired AI Meets AGI](https://mlsanity.com/paper/2303.15935/) (2023)
- [Polisis: Automated Analysis and Presentation of Privacy Policies Using Deep Learning](https://mlsanity.com/paper/1802.02561/) (2018)
- [Privacy Policies over Time: Curation and Analysis of a Million-Document Dataset](https://mlsanity.com/paper/2008.09159/) (2020)
- [The Privacy Policy Landscape After the GDPR](https://mlsanity.com/paper/1809.08396/) (2018)
- [We Value Your Privacy ... Now Take Some Cookies: Measuring the GDPR's Impact on Web Privacy](https://mlsanity.com/paper/1808.05096/) (2018)

## Built on by (most cited first, 16 of 16)

- [Revolutionizing Finance with LLMs: An Overview of Applications and Insights](https://mlsanity.com/paper/2401.11641/) (2024)
- [CBR-RAG: Case-Based Reasoning for Retrieval Augmented Generation in LLMs for Legal Question Answering](https://mlsanity.com/paper/2404.04302/) (2024)
- [Holistic Evaluation of GPT-4V for Biomedical Imaging](https://mlsanity.com/paper/2312.05256/) (2023)
- [Large Language Models: A New Approach for Privacy Policy Analysis at Scale](https://mlsanity.com/paper/2405.20900/) (2024)
- [Large Language Models are legal but they are not: Making the case for a powerful LegalLLM](https://mlsanity.com/paper/2311.08890/) (2023)
- [Metamorphic Testing of Large Language Models for Natural Language Processing](https://mlsanity.com/paper/2511.02108/) (2025)
- [Exploring the Nexus of Large Language Models and Legal Systems: A Short Survey](https://mlsanity.com/paper/2404.00990/) (2024)
- [Large Language Models for Bioinformatics](https://mlsanity.com/paper/2501.06271/) (2025)
- [Entailment-Driven Privacy Policy Classification with LLMs](https://mlsanity.com/paper/2409.16621/) (2024)
- [Let's Measure the Elephant in the Room: Facilitating Personalized Automated Analysis of Privacy Policies at Scale](https://mlsanity.com/paper/2507.14214/) (2025)
- [PrivPRISM: Automatically Detecting Discrepancies Between Google Play Data Safety Declarations and Developer Privacy Policies](https://mlsanity.com/paper/2603.09214/) (2026)
- [Generating Privacy Stories From Software Documentation](https://mlsanity.com/paper/2506.23014/) (2025)
- [An LLM-enabled semantic-centric framework to consume privacy policies](https://mlsanity.com/paper/2509.01716/) (2025)
- [PrivSTRUCT: Untangling Data Purpose Compliance of Privacy Policies in Google Play Store](https://mlsanity.com/paper/2604.22157/) (2026)
- [Customized Generative AI Agent for Transportation Engineering Practice: A Development and Continued Pre-training Guideline](https://mlsanity.com/paper/2606.29014/) (2026)
- [SoK: From Generation to Consumption of Privacy Documents in Software Systems](https://mlsanity.com/paper/2608.12511/) (2026)

## Abstract (arXiv)

Privacy policies serve as the primary conduit through which online service providers inform users about their data collection and usage procedures. However, in a bid to be comprehensive and mitigate legal risks, these policy documents are often quite verbose. In practical use, users tend to click the Agree button directly rather than reading them carefully. This practice exposes users to risks of privacy leakage and legal issues. Recently, the advent of Large Language Models (LLM) such as ChatGPT and GPT-4 has opened new possibilities for text analysis, especially for lengthy documents like privacy policies. In this study, we investigate a privacy policy text analysis framework PolicyGPT based on the LLM. This framework was tested using two datasets. The first dataset comprises of privacy policies from 115 websites, which were meticulously annotated by legal experts, categorizing each segment into one of 10 classes. The second dataset consists of privacy policies from 304 popular mobile applications, with each sentence manually annotated and classified into one of another 10 categories. Under zero-shot learning conditions, PolicyGPT demonstrated robust performance. For the first dataset, it achieved an accuracy rate of 97%, while for the second dataset, it attained an 87% accuracy rate, surpassing that of the baseline machine learning and neural network models.