# AI-enabled Automation for Completeness Checking of Privacy Policies

arXiv: https://arxiv.org/abs/2106.05688 · HTML: https://mlsanity.com/paper/2106.05688/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 79 references that are in the map.
- First posted on arXiv 10 Jun 2021.

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings](https://mlsanity.com/paper/1909.00512/) (2019)
- [All-but-the-Top: Simple and Effective Postprocessing for Word Representations](https://mlsanity.com/paper/1702.01417/) (2017)
- [When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?](https://mlsanity.com/paper/1804.06323/) (2018)
- [CLAUDETTE: an Automated Detector of Potentially Unfair Clauses in Online Terms of Service](https://mlsanity.com/paper/1805.01217/) (2018)
- [Towards Universal Paraphrastic Sentence Embeddings](https://mlsanity.com/paper/1511.08198/) (2015)
- [Are Word Embedding-based Features Useful for Sarcasm Detection?](https://mlsanity.com/paper/1610.00883/) (2016)
- [An Empirical Evaluation of GDPR Compliance Violations in Android mHealth Apps](https://mlsanity.com/paper/2008.05864/) (2020)
- [Designing Privacy-aware Internet of Things Applications](https://mlsanity.com/paper/1703.03892/) (2017)
- [Model Driven Engineering for Data Protection and Privacy: Application and Experience with GDPR](https://mlsanity.com/paper/2007.12046/) (2020)

## Built on by (most cited first, 7 of 7)

- [Enhancing Legal Compliance and Regulation Analysis with Large Language Models](https://mlsanity.com/paper/2404.17522/) (2024)
- [Helping Johnny Make Sense of Privacy Policies with LLMs](https://mlsanity.com/paper/2501.16033/) (2025)
- [A Multi-solution Study on GDPR AI-enabled Completeness Checking of DPAs](https://mlsanity.com/paper/2311.13881/) (2023)
- [A Human-in-the-Loop Approach for Information Extraction from Privacy Policies under Data Scarcity](https://mlsanity.com/paper/2305.15006/) (2023)
- [Legal Requirements Analysis](https://mlsanity.com/paper/2311.13871/) (2023)
- [Legally Binding but Unfair? Towards Assessing Fairness of Privacy Policies](https://mlsanity.com/paper/2403.08115/) (2024)
- [Privacy Washing: Detecting Internal Contradictions in Privacy Policies](https://mlsanity.com/paper/2609.02055/) (2026)

## Abstract (arXiv)

Technological advances in information sharing have raised concerns about data protection. Privacy policies contain privacy-related requirements about how the personal data of individuals will be handled by an organization or a software system (e.g., a web service or an app). In Europe, privacy policies are subject to compliance with the General Data Protection Regulation (GDPR). A prerequisite for GDPR compliance checking is to verify whether the content of a privacy policy is complete according to the provisions of GDPR. Incomplete privacy policies might result in large fines on violating organization as well as incomplete privacy-related software specifications. Manual completeness checking is both time-consuming and error-prone. In this paper, we propose AI-based automation for the completeness checking of privacy policies. Through systematic qualitative methods, we first build two artifacts to characterize the privacy-related provisions of GDPR, namely a conceptual model and a set of completeness criteria. Then, we develop an automated solution on top of these artifacts by leveraging a combination of natural language processing and supervised machine learning. Specifically, we identify the GDPR-relevant information content in privacy policies and subsequently check them against the completeness criteria. To evaluate our approach, we collected 234 real privacy policies from the fund industry. Over a set of 48 unseen privacy policies, our approach detected 300 of the total of 334 violations of some completeness criteria correctly, while producing 23 false positives. The approach thus has a precision of 92.9% and recall of 89.8%. Compared to a baseline that applies keyword search only, our approach results in an improvement of 24.5% in precision and 38% in recall.