# Privacy Policy Analysis through Prompt Engineering for LLMs

arXiv: https://arxiv.org/abs/2409.14879 · HTML: https://mlsanity.com/paper/2409.14879/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 71 references that are in the map.
- First posted on arXiv 23 Sep 2024.

## Builds on (most cited first, 13 of 13)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [Large Language Models Are Human-Level Prompt Engineers](https://mlsanity.com/paper/2211.01910/) (2022)
- [Design Guidelines for Prompt Engineering Text-to-Image Generative Models](https://mlsanity.com/paper/2109.06977/) (2021)
- [Question Answering for Privacy Policies: Combining Computational and Legal Perspectives](https://mlsanity.com/paper/1911.00841/) (2019)
- [Intent Classification and Slot Filling for Privacy Policies](https://mlsanity.com/paper/2101.00123/) (2021)
- [Large Language Models: A New Approach for Privacy Policy Analysis at Scale](https://mlsanity.com/paper/2405.20900/) (2024)
- [PoliGraph: Automated Privacy Policy Analysis using Knowledge Graphs (Journal Version)](https://mlsanity.com/paper/2210.06746/) (2022)
- [Prospects for inconsistency detection using large language models and sheaves](https://mlsanity.com/paper/2401.16713/) (2024)
- [Modeling Language Vagueness in Privacy Policies using Deep Neural Networks](https://mlsanity.com/paper/1805.10393/) (2018)
- [Beyond The Text: Analysis of Privacy Statements through Syntactic and Semantic Role Labeling](https://mlsanity.com/paper/2010.00678/) (2020)

## Built on by (most cited first, 6 of 6)

- [A Survey of Context Engineering for Large Language Models](https://mlsanity.com/paper/2507.13334/) (2025)
- [Using LLMs for Automated Privacy Policy Analysis: Prompt Engineering, Fine-Tuning and Explainability](https://mlsanity.com/paper/2503.16516/) (2025)
- [Interpretation Gaps in LLM-Assisted Comprehension of Privacy Documents](https://mlsanity.com/paper/2503.12225/) (2025)
- [LADFA: A Framework of Using Large Language Models and Retrieval-Augmented Generation for Personal Data Flow Analysis in Privacy Policies](https://mlsanity.com/paper/2601.10413/) (2026)
- [PolicyKG: An Agentic LLM Pipeline for Translating Institutional Policies into SHACL Knowledge Graphs](https://mlsanity.com/paper/2608.09028/) (2026)
- [Policy-as-Skill: Governed LLM Decision Support with Evidence, Deterministic Control, and Audit](https://mlsanity.com/paper/2609.27087/) (2026)

## Abstract (arXiv)

Privacy policies are often obfuscated by their complexity, which impedes transparency and informed consent. Conventional machine learning approaches for automatically analyzing these policies demand significant resources and substantial domain-specific training, causing adaptability issues. Moreover, they depend on extensive datasets that may require regular maintenance due to changing privacy concerns. In this paper, we propose, apply, and assess PAPEL (Privacy Policy Analysis through Prompt Engineering for LLMs), a framework harnessing the power of Large Language Models (LLMs) through prompt engineering to automate the analysis of privacy policies. PAPEL aims to streamline the extraction, annotation, and summarization of information from these policies, enhancing their accessibility and comprehensibility without requiring additional model training. By integrating zero-shot, one-shot, and few-shot learning approaches and the chain-of-thought prompting in creating predefined prompts and prompt templates, PAPEL guides LLMs to efficiently dissect, interpret, and synthesize the critical aspects of privacy policies into user-friendly summaries. We demonstrate the effectiveness of PAPEL with two applications: (i) annotation and (ii) contradiction analysis. We assess the ability of several LLaMa and GPT models to identify and articulate data handling practices, offering insights comparable to existing automated analysis approaches while reducing training efforts and increasing the adaptability to new analytical needs. The experiments demonstrate that the LLMs PAPEL utilizes (LLaMA and Chat GPT models) achieve robust performance in privacy policy annotation, with F1 scores reaching 0.8 and above (using the OPP-115 gold standard), underscoring the effectiveness of simpler prompts across various advanced language models.