mlsanityData to 14 Oct 2026

Helping Johnny Make Sense of Privacy Policies with LLMs

arXiv 2501.16033 ↗First posted 27 Jan 2025cs.HC cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Understanding and engaging with privacy policies is crucial for online privacy, yet these documents remain notoriously complex and difficult to navigate. We present PRISMe, an interactive browser extension that combines LLM-based policy assessment with a dashboard and customizable chat interface, enabling users to skim quick overviews or explore policy details in depth while browsing. We conduct a user study (N=22) with participants of diverse privacy knowledge to investigate how users interpret the tool's explanations and how it shapes their engagement with privacy policies, identifying distinct interaction patterns. Participants valued the clear overviews and conversational depth, but flagged some issues, particularly adversarial robustness and hallucination risks. Thus, we investigate how a retrieval-augmented generation (RAG) approach can alleviate issues by re-running the chat queries from the study. Our findings surface design challenges as well as technical trade-offs, contributing actionable insights for developing future user-centered, trustworthy privacy policy analysis tools.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 25 of its 105 references that are in the map. First posted on arXiv 27 Jan 2025.

Builds on

25 of 105 references.

References · 25

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  4. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 452
  5. 2017Wild Patterns: Ten Years After the Rise of Adversarial Machine Learningcited by 414
  6. 2023In-Context Impersonation Reveals Large Language Models' Strengths and Biasescited by 155
  7. 2024Is LLM-as-a-Judge Robust? Investigating Universal Adversarial Attacks on Zero-shot LLM Assessmentcited by 126
  8. 2023The Metacognitive Demands and Opportunities of Generative AIcited by 87
  9. 2024"I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trustcited by 67
  10. 2018Polisis: Automated Analysis and Presentation of Privacy Policies Using Deep Learningcited by 50
  11. 2024CLEAR: Towards Contextual LLM-Empowered Privacy Policy Analysis and Risk Generation for Large Language Model Applicationscited by 33
  12. 2023Deepfakes, Phrenology, Surveillance, and More! A Taxonomy of AI Privacy Riskscited by 31
13 more
  1. 2024Large Language Models: A New Approach for Privacy Policy Analysis at Scalecited by 14
  2. 2024Empowering Users in Digital Privacy Management through Interactive LLM-Based Agentscited by 9
  3. 2021AI-enabled Automation for Completeness Checking of Privacy Policiescited by 7
  4. 2023GenAIPABench: A Benchmark for Generative AI-based Privacy Assistantscited by 7
  5. 2020TILT: A GDPR-Aligned Transparency Information Language and Toolkit for Practical Privacy Engineeringcited by 6
  6. 2023Truth in Motion: The Unprecedented Risks and Opportunities of Extended Reality Motion Datacited by 4
  7. 2022Feedback Exchange and Online Affinity: A Case Study of Online Fanfiction Writerscited by 3
  8. 2022Exploring the Online Micro-targeting Practices of Small, Medium, and Large Businessescited by 2
  9. 2023Enabling Versatile Privacy Interfaces Using Machine-Readable Transparency Informationcited by 2
  10. 2023A Human-in-the-Loop Approach for Information Extraction from Privacy Policies under Data Scarcitycited by 2
  11. 2024Identifying Privacy Personascited by 2
  12. 2021Bolder is Better: Raising User Awareness through Salient and Concise Privacy Noticescited by 1
  13. 2024Legally Binding but Unfair? Towards Assessing Fairness of Privacy Policiescited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 3.

5 papers.

  1. 2025Position: Privacy Is Not Just Memorization!cited by 17
  2. 2025A Comprehensive Review on Harnessing Large Language Models to Overcome Recommender System Challengescited by 3
  3. 2026Understanding U.S. Users' Security and Privacy Transparency Needs for Consumer-Facing Generative AIcited by 0
  4. 2026The Privacy Guardian Agent: Towards Trustworthy AI Privacy Agentscited by 0
  5. 2026GuardianAgent: Policy-Conditioned Risk-Adaptive Anonymization with Verified Adversarial Escalationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Explainable AI in Usable Privacy and Security: Challenges and Opportunities
  2. 2025A Longitudinal Measurement of Privacy Policy Evolution for Large Language Models
  3. 2026Privacy Control in Conversational LLM Platforms: A Walkthrough Study
  4. 2026LADFA: A Framework of Using Large Language Models and Retrieval-Augmented Generation for Personal Data Flow Analysis in Privacy Policies
  5. 2026PrivacyReasoner: Can LLM Emulate a Human-like Privacy Mind?
  6. 2023"It's a Fair Game", or Is It? Examining How Users Navigate Disclosure Risks and Benefits When Using LLM-Based Conversational Agents
  7. 2025User Behavior Analysis in Privacy Protection with Large Language Models: A Study on Privacy Preferences with Limited Data
  8. 2024Privacy Policy Analysis through Prompt Engineering for LLMs