# AHA!: Facilitating AI Impact Assessment by Generating Examples of Harms

arXiv: https://arxiv.org/abs/2306.03280 · HTML: https://mlsanity.com/paper/2306.03280/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jun 2023.

## Built on by (most cited first, 26 of 26)

- [Towards Trustworthy GUI Agents: A Survey](https://mlsanity.com/paper/2503.23434/) (2025)
- [Farsight: Fostering Responsible AI Awareness During AI Application Prototyping](https://mlsanity.com/paper/2402.15350/) (2024)
- [AI on My Shoulder: Supporting Emotional Labor in Front-Office Roles with an LLM-based Empathetic Coworker](https://mlsanity.com/paper/2411.02408/) (2024)
- [Co-designing an AI Impact Assessment Report Template with AI Practitioners and AI Compliance Experts](https://mlsanity.com/paper/2407.17374/) (2024)
- [Particip-AI: A Democratic Surveying Framework for Anticipating Future AI Use Cases, Harms and Benefits](https://mlsanity.com/paper/2403.14791/) (2024)
- [Responsible AI Research Needs Impact Statements Too](https://mlsanity.com/paper/2311.11776/) (2023)
- [From Hazard Identification to Controller Design: Proactive and LLM-Supported Safety Engineering for ML-Powered Systems](https://mlsanity.com/paper/2502.07974/) (2025)
- [BLIP: Facilitating the Exploration of Undesirable Consequences of Digital Technologies](https://mlsanity.com/paper/2405.06783/) (2024)
- [Simulating Policy Impacts: Developing a Generative Scenario Writing Method to Evaluate the Perceived Effects of Regulation](https://mlsanity.com/paper/2405.09679/) (2024)
- [Privy: Envisioning and Mitigating Privacy Risks for Consumer-facing AI Product Concepts](https://mlsanity.com/paper/2509.23525/) (2025)
- [The AI Model Risk Catalog: What Developers and Researchers Miss About Real-World AI Harms](https://mlsanity.com/paper/2508.16672/) (2025)
- [How Well Can LLM Agents Simulate End-User Security and Privacy Attitudes and Behaviors?](https://mlsanity.com/paper/2602.18464/) (2026)
- [Responsible AI Considerations in Text Summarization Research: A Review of Current Practices](https://mlsanity.com/paper/2311.11103/) (2023)
- [Ethics Whitepaper: Whitepaper on Ethical Research into Large Language Models](https://mlsanity.com/paper/2410.19812/) (2024)
- [AI LEGO: Scaffolding Cross-Functional Collaboration in Industrial Responsible AI Practices during Early Design Stages](https://mlsanity.com/paper/2505.10300/) (2025)
- [What Is the Point of Equality in Machine Learning Fairness? Beyond Equality of Opportunity](https://mlsanity.com/paper/2506.16782/) (2025)
- [ECHO: A Participatory Framework for Bias-Anchored AI Harm Anticipation](https://mlsanity.com/paper/2512.03068/) (2025)
- [Interrogating Design Homogenization in Web Vibe Coding](https://mlsanity.com/paper/2603.13036/) (2026)
- [Evaluating the Capabilities of LLMs for Supporting Anticipatory Impact Assessment](https://mlsanity.com/paper/2401.18028/) (2024)
- [Towards Leveraging News Media to Support Impact Assessment of AI Technologies](https://mlsanity.com/paper/2411.02536/) (2024)
- [Telling Speculative Stories to Help Humans Imagine the Harms of Healthcare AI](https://mlsanity.com/paper/2510.14718/) (2025)
- [Informing AI Policy Assessment using Large-Scale Simulation of Interventions](https://mlsanity.com/paper/2605.27395/) (2026)
- [The Perils of Agency: How Developers Perceive, Prioritize, and Address Risks in Agentic AI Products](https://mlsanity.com/paper/2606.15485/) (2026)
- [Unaccountable Delegation, Fading Skills: Mapping the Risks of Workplace AI Agents](https://mlsanity.com/paper/2608.08601/) (2026)
- [Ethics Training Agents: Facilitating Group-Based Ethics Education with Role-Playing and Discussion for Ethical Reflection and Exploration](https://mlsanity.com/paper/2609.11529/) (2026)
- [Small-world Networks of Agents Brainstorm AI Risks to Support Ideation](https://mlsanity.com/paper/2609.24859/) (2026)

## Abstract (arXiv)

While demands for change and accountability for harmful AI consequences mount, foreseeing the downstream effects of deploying AI systems remains a challenging task. We developed AHA! (Anticipating Harms of AI), a generative framework to assist AI practitioners and decision-makers in anticipating potential harms and unintended consequences of AI systems prior to development or deployment. Given an AI deployment scenario, AHA! generates descriptions of possible harms for different stakeholders. To do so, AHA! systematically considers the interplay between common problematic AI behaviors as well as their potential impacts on different stakeholders, and narrates these conditions through vignettes. These vignettes are then filled in with descriptions of possible harms by prompting crowd workers and large language models. By examining 4113 harms surfaced by AHA! for five different AI deployment scenarios, we found that AHA! generates meaningful examples of harms, with different problematic AI behaviors resulting in different types of harms. Prompting both crowds and a large language model with the vignettes resulted in more diverse examples of harms than those generated by either the crowd or the model alone. To gauge AHA!'s potential practical utility, we also conducted semi-structured interviews with responsible AI professionals (N=9). Participants found AHA!'s systematic approach to surfacing harms important for ethical reflection and discovered meaningful stakeholders and harms they believed they would not have thought of otherwise. Participants, however, differed in their opinions about whether AHA! should be used upfront or as a secondary-check and noted that AHA! may shift harm anticipation from an ideation problem to a potentially demanding review problem. Drawing on our results, we discuss design implications of building tools to help practitioners envision possible harms.