# Word-level Annotation of GDPR Transparency Compliance in Privacy Policies using Large Language Models

arXiv: https://arxiv.org/abs/2503.10727 · HTML: https://mlsanity.com/paper/2503.10727/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 61 references that are in the map.
- First posted on arXiv 13 Mar 2025.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [On Faithfulness and Factuality in Abstractive Summarization](https://mlsanity.com/paper/2005.00661/) (2020)
- [Phi-4 Technical Report](https://mlsanity.com/paper/2412.08905/) (2024)
- [Polisis: Automated Analysis and Presentation of Privacy Policies Using Deep Learning](https://mlsanity.com/paper/1802.02561/) (2018)
- [Privacy at Scale: Introducing the PrivaSeer Corpus of Web Privacy Policies](https://mlsanity.com/paper/2004.11131/) (2020)
- [The Privacy Policy Landscape After the GDPR](https://mlsanity.com/paper/1809.08396/) (2018)
- [Large Language Models: A New Approach for Privacy Policy Analysis at Scale](https://mlsanity.com/paper/2405.20900/) (2024)
- [Large Language Models For Text Classification: Case Study And Comprehensive Review](https://mlsanity.com/paper/2501.08457/) (2025)
- [Are iPhones Really Better for Privacy? Comparative Study of iOS and Android Apps](https://mlsanity.com/paper/2109.13722/) (2021)
- [Third Party Tracking in the Mobile Ecosystem](https://mlsanity.com/paper/1804.03603/) (2018)
- [A Fait Accompli? An Empirical Study into the Absence of Consent to Third-Party Tracking in Android Apps](https://mlsanity.com/paper/2106.09407/) (2021)
- [A Qualitative Analysis Framework for mHealth Privacy Practices](https://mlsanity.com/paper/2405.17971/) (2024)

## Built on by (most cited first, 1 of 1)

- [Addressing Labelled Data Scarcity: Taxonomy-Agnostic Annotation of PII Values in HTTP Traffic using LLMs](https://mlsanity.com/paper/2605.06305/) (2026)

## Abstract (arXiv)

Ensuring transparency of data practices related to personal information is a core requirement of the General Data Protection Regulation (GDPR). However, large-scale compliance assessment remains challenging due to the complexity and diversity of privacy policy language. Manual audits are labour-intensive and inconsistent, while current automated methods often lack the granularity required to capture nuanced transparency disclosures. In this paper, we present a modular large language model (LLM)-based pipeline for fine-grained word-level annotation of privacy policies with respect to GDPR transparency requirements. Our approach integrates LLM-driven annotation with passage-level classification, retrieval-augmented generation, and a self-correction mechanism to deliver scalable, context-aware annotations across 21 GDPR-derived transparency requirements. To support empirical evaluation, we compile a corpus of 703,791 English-language privacy policies and generate a ground-truth sample of 200 manually annotated policies based on a comprehensive, GDPR-aligned annotation scheme. We propose a two-tiered evaluation methodology capturing both passage-level classification and span-level annotation quality and conduct a comparative analysis of seven state-of-the-art LLMs on two annotation schemes, including the widely used OPP-115 dataset. The results of our evaluation show that decomposing the annotation task and integrating targeted retrieval and classification components significantly improve annotation accuracy, particularly for well-structured requirements. Our work provides new empirical resources and methodological foundations for advancing automated transparency compliance assessment at scale.