# Privy: Envisioning and Mitigating Privacy Risks for Consumer-facing AI Product Concepts

arXiv: https://arxiv.org/abs/2509.23525 · HTML: https://mlsanity.com/paper/2509.23525/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 83 references that are in the map.
- First posted on arXiv 27 Sep 2025.

## Builds on (most cited first, 22 of 22)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Generative artificial intelligence enhances creativity but reduces the diversity of novel content](https://mlsanity.com/paper/2312.00506/) (2023)
- [From ChatGPT to ThreatGPT: Impact of Generative AI in Cybersecurity and Privacy](https://mlsanity.com/paper/2307.00691/) (2023)
- [Towards a Visual Privacy Advisor: Understanding and Predicting Privacy Risks in Images](https://mlsanity.com/paper/1703.10660/) (2017)
- [Dislocated Accountabilities in the AI Supply Chain: Modularity and Developers' Notions of Responsibility](https://mlsanity.com/paper/2209.09780/) (2022)
- [Deepfakes, Phrenology, Surveillance, and More! A Taxonomy of AI Privacy Risks](https://mlsanity.com/paper/2310.07879/) (2023)
- [How Knowledge Workers Think Generative AI Will (Not) Transform Their Industries](https://mlsanity.com/paper/2310.06778/) (2023)
- [Farsight: Fostering Responsible AI Awareness During AI Application Prototyping](https://mlsanity.com/paper/2402.15350/) (2024)
- [AHA!: Facilitating AI Impact Assessment by Generating Examples of Harms](https://mlsanity.com/paper/2306.03280/) (2023)
- [Walking the Walk of AI Ethics: Organizational Challenges and the Individualization of Risk among Ethics Entrepreneurs](https://mlsanity.com/paper/2305.09573/) (2023)
- [ExploreGen: Large Language Models for Envisioning the Uses and Risks of AI Technologies](https://mlsanity.com/paper/2407.12454/) (2024)
- [AI Should Challenge, Not Obey](https://mlsanity.com/paper/2411.02263/) (2024)
- [Co-designing an AI Impact Assessment Report Template with AI Practitioners and AI Compliance Experts](https://mlsanity.com/paper/2407.17374/) (2024)
- [Ethics in the Age of AI: An Analysis of AI Practitioners' Awareness and Challenges](https://mlsanity.com/paper/2307.10057/) (2023)
- [Antagonistic AI](https://mlsanity.com/paper/2402.07350/) (2024)
- [The "Who", "What", and "How" of Responsible AI Governance: A Systematic Review and Meta-Analysis of (Actor, Stage)-Specific Tools](https://mlsanity.com/paper/2502.13294/) (2025)
- [Privacy Risks of General-Purpose AI Systems: A Foundation for Investigating Practitioner Perspectives](https://mlsanity.com/paper/2407.02027/) (2024)
- [Evaluating Privacy Perceptions, Experience, and Behavior of Software Development Teams](https://mlsanity.com/paper/2404.01283/) (2024)
- [Supporting Industry Computing Researchers in Assessing, Articulating, and Addressing the Potential Negative Societal Impact of Their Work](https://mlsanity.com/paper/2408.01057/) (2024)
- [Privacy Engineering Meets Software Engineering. On the Challenges of Engineering Privacy ByDesign](https://mlsanity.com/paper/2007.08613/) (2020)
- [Ontologies for Privacy Requirements Engineering: A Systematic Literature Review](https://mlsanity.com/paper/1611.10097/) (2016)

## Built on by (most cited first, 4 of 4)

- [How Well Can LLM Agents Simulate End-User Security and Privacy Attitudes and Behaviors?](https://mlsanity.com/paper/2602.18464/) (2026)
- [Understanding U.S. Users' Security and Privacy Transparency Needs for Consumer-Facing Generative AI](https://mlsanity.com/paper/2604.17270/) (2026)
- [The Perils of Agency: How Developers Perceive, Prioritize, and Address Risks in Agentic AI Products](https://mlsanity.com/paper/2606.15485/) (2026)
- [A Human-LLM Teaming Framework for Privacy Risk Analysis: An Illustration with CBDC-Based Welfare Schemes](https://mlsanity.com/paper/2608.16461/) (2026)

## Abstract (arXiv)

AI creates and exacerbates privacy risks, yet practitioners lack effective resources to identify and mitigate these risks. We present Privy, a tool that guides practitioners without privacy expertise through structured privacy impact assessments to: (i) identify relevant risks in novel AI product concepts, and (ii) propose appropriate mitigations. Privy was shaped by a formative study with 11 practitioners, which informed two versions -- one LLM-powered, the other template-based. We evaluated these two versions of Privy through a between-subjects, controlled study with 24 separate practitioners, whose assessments were reviewed by 13 independent privacy experts. Results show that Privy helps practitioners produce privacy assessments that experts deemed high quality: practitioners identified relevant risks and proposed appropriate mitigation strategies. These effects were augmented in the LLM-powered version. Practitioners themselves rated Privy as being useful and usable, and their feedback illustrates how it helps overcome long-standing awareness, motivation, and ability barriers in privacy work.