# Question Answering for Privacy Policies: Combining Computational and Legal Perspectives

arXiv: https://arxiv.org/abs/1911.00841 · HTML: https://mlsanity.com/paper/1911.00841/

## Key facts

- Cited by 67 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Nov 2019.

## Built on by (most cited first, 30 of 67)

- [A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers](https://mlsanity.com/paper/2105.03011/) (2021)
- [LexGLUE: A Benchmark Dataset for Legal Language Understanding in English](https://mlsanity.com/paper/2110.00976/) (2021)
- [LegalBench-RAG: A Benchmark for Retrieval-Augmented Generation in the Legal Domain](https://mlsanity.com/paper/2408.10343/) (2024)
- [The Art of Saying No: Contextual Noncompliance in Language Models](https://mlsanity.com/paper/2407.12043/) (2024)
- [Multi-LexSum: Real-World Summaries of Civil Rights Lawsuits at Multiple Granularities](https://mlsanity.com/paper/2206.10883/) (2022)
- [LEVEN: A Large-Scale Chinese Legal Event Detection Dataset](https://mlsanity.com/paper/2203.08556/) (2022)
- [Interpretable Long-Form Legal Question Answering with Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2309.17050/) (2023)
- [ConditionalQA: A Complex Reading Comprehension Dataset with Conditional Answers](https://mlsanity.com/paper/2110.06884/) (2021)
- [Learning to Generate Instruction Tuning Datasets for Zero-Shot Task Adaptation](https://mlsanity.com/paper/2402.18334/) (2024)
- [NoiseQA: Challenge Set Evaluation for User-Centric Question Answering](https://mlsanity.com/paper/2102.08345/) (2021)
- [Intent Classification and Slot Filling for Privacy Policies](https://mlsanity.com/paper/2101.00123/) (2021)
- [VNHSGE: VietNamese High School Graduation Examination Dataset for Large Language Models](https://mlsanity.com/paper/2305.12199/) (2023)
- [Large Language Models: A New Approach for Privacy Policy Analysis at Scale](https://mlsanity.com/paper/2405.20900/) (2024)
- [Towards Reliable Retrieval in RAG Systems for Large Legal Datasets](https://mlsanity.com/paper/2510.06999/) (2025)
- [A Survey on Legal Question Answering Systems](https://mlsanity.com/paper/2110.07333/) (2021)
- [Retrieval Enhanced Data Augmentation for Question Answering on Privacy Policies](https://mlsanity.com/paper/2204.08952/) (2022)
- [Legal Question-Answering in the Indian Context: Efficacy, Challenges, and Potential of Modern AI Models](https://mlsanity.com/paper/2309.14735/) (2023)
- [LawInstruct: A Resource for Studying Language Model Adaptation to the Legal Domain](https://mlsanity.com/paper/2404.02127/) (2024)
- [PositionID: LLMs can Control Lengths, Copy and Paste with Explicit Positional Awareness](https://mlsanity.com/paper/2410.07035/) (2024)
- [Empowering Users in Digital Privacy Management through Interactive LLM-Based Agents](https://mlsanity.com/paper/2410.11906/) (2024)
- [DatasetResearch: Benchmarking Agent Systems for Demand-Driven Dataset Discovery](https://mlsanity.com/paper/2508.06960/) (2025)
- [LexSumm and LexT5: Benchmarking and Modeling Legal Summarization Tasks in English](https://mlsanity.com/paper/2410.09527/) (2024)
- [Lawma: The Power of Specialization for Legal Annotation](https://mlsanity.com/paper/2407.16615/) (2024)
- [Privacy Policy Analysis through Prompt Engineering for LLMs](https://mlsanity.com/paper/2409.14879/) (2024)
- [Privacy Policy Question Answering Assistant: A Query-Guided Extractive Summarization Approach](https://mlsanity.com/paper/2109.14638/) (2021)
- [PLUE: Language Understanding Evaluation Benchmark for Privacy Policies in English](https://mlsanity.com/paper/2212.10011/) (2022)
- [ATLAS: Automatically Detecting Discrepancies Between Privacy Policies and Privacy Labels](https://mlsanity.com/paper/2306.09247/) (2023)
- [Automating IRAC Analysis in Malaysian Contract Law using a Semi-Structured Knowledge Base](https://mlsanity.com/paper/2406.13217/) (2024)
- [Ranking Free RAG: Replacing Re-ranking with Selection in RAG for Sensitive Domains](https://mlsanity.com/paper/2505.16014/) (2025)
- [GRAF: Graph Retrieval Augmented by Facts for Romanian Legal Multi-Choice Question Answering](https://mlsanity.com/paper/2412.04119/) (2024)

## Abstract (arXiv)

Privacy policies are long and complex documents that are difficult for users to read and understand, and yet, they have legal effects on how user data is collected, managed and used. Ideally, we would like to empower users to inform themselves about issues that matter to them, and enable them to selectively explore those issues. We present PrivacyQA, a corpus consisting of 1750 questions about the privacy policies of mobile applications, and over 3500 expert annotations of relevant answers. We observe that a strong neural baseline underperforms human performance by almost 0.3 F1 on PrivacyQA, suggesting considerable room for improvement for future systems. Further, we use this dataset to shed light on challenges to question answerability, with domain-general implications for any question answering system. The PrivacyQA corpus offers a challenging corpus for question answering, with genuine real-world utility.