# An LLM-enabled semantic-centric framework to consume privacy policies

arXiv: https://arxiv.org/abs/2509.01716 · HTML: https://mlsanity.com/paper/2509.01716/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 42 references that are in the map.
- First posted on arXiv 1 Sep 2025.

## Builds on (most cited first, 22 of 22)

- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems](https://mlsanity.com/paper/1905.00537/) (2019)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [Unifying Large Language Models and Knowledge Graphs: A Roadmap](https://mlsanity.com/paper/2306.08302/) (2023)
- [A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications](https://mlsanity.com/paper/2402.07927/) (2024)
- [LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models](https://mlsanity.com/paper/2308.11462/) (2023)
- [A Survey on Deep Learning for Named Entity Recognition](https://mlsanity.com/paper/1812.09449/) (2018)
- [Revisiting Relation Extraction in the era of Large Language Models](https://mlsanity.com/paper/2305.05003/) (2023)
- [Question Answering for Privacy Policies: Combining Computational and Legal Perspectives](https://mlsanity.com/paper/1911.00841/) (2019)
- [Tranco: A Research-Oriented Top Sites Ranking Hardened Against Manipulation](https://mlsanity.com/paper/1806.01156/) (2018)
- [Polisis: Automated Analysis and Presentation of Privacy Policies Using Deep Learning](https://mlsanity.com/paper/1802.02561/) (2018)
- [PolicyQA: A Reading Comprehension Dataset for Privacy Policies](https://mlsanity.com/paper/2010.02557/) (2020)
- [Privacy Policies over Time: Curation and Analysis of a Million-Document Dataset](https://mlsanity.com/paper/2008.09159/) (2020)
- [A Comprehensive Survey on Relation Extraction: Recent Advances and New Frontiers](https://mlsanity.com/paper/2306.02051/) (2023)
- [Intent Classification and Slot Filling for Privacy Policies](https://mlsanity.com/paper/2101.00123/) (2021)
- [PolicyGPT: Automated Analysis of Privacy Policies with Large Language Models](https://mlsanity.com/paper/2309.10238/) (2023)
- [Large Language Models: A New Approach for Privacy Policy Analysis at Scale](https://mlsanity.com/paper/2405.20900/) (2024)
- [PoliGraph: Automated Privacy Policy Analysis using Knowledge Graphs (Journal Version)](https://mlsanity.com/paper/2210.06746/) (2022)
- [Ask the Experts: What Should Be on an IoT Privacy and Security Label?](https://mlsanity.com/paper/2002.04631/) (2020)
- [Data Privacy Vocabulary (DPV) -- Version 2](https://mlsanity.com/paper/2404.13426/) (2024)
- [Perennial Semantic Data Terms of Use for Decentralized Web](https://mlsanity.com/paper/2403.07587/) (2024)

## Abstract (arXiv)

In modern times, people have numerous online accounts, but they rarely read the Terms of Service or Privacy Policy of those sites, despite claiming otherwise, due to the practical difficulty in comprehending them. The mist of data privacy practices forms a major barrier for user-centred Web approaches, and for data sharing and reusing in an agentic world. Existing research proposed methods for using formal languages and reasoning for verifying the compliance of a specified policy, as a potential cure for ignoring privacy policies. However, a critical gap remains in the creation or acquisition of such formal policies at scale. We present a semantic-centric approach for using state-of-the-art large language models (LLM), to automatically identify key information about privacy practices from privacy policies, and construct mathitPr2mathitGraph, knowledge graph with grounding from Data Privacy Vocabulary (DPV) for privacy practices, to support downstream tasks. Along with the pipeline, the mathitPr2mathitGraph for the top-100 popular websites is also released as a public resource, by using the pipeline for analysis. We also demonstrate how the mathitPr2mathitGraph can be used to support downstream tasks by constructing formal policy representations such as Open Digital Right Language (ODRL) or perennial semantic Data Terms of Use (psDToU). To evaluate the technology capability, we enriched the Policy-IE dataset by employing legal experts to create custom annotations. We benchmarked the performance of different large language models for our pipeline and verified their capabilities. Overall, they shed light on the possibility of large-scale analysis of online services' privacy practices, as a promising direction to audit the Web and the Internet. We release all datasets and source code as public resources to facilitate reuse and improvement.