# Overcoming Failures of Imagination in AI Infused System Development and Deployment

arXiv: https://arxiv.org/abs/2011.13416 · HTML: https://mlsanity.com/paper/2011.13416/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Nov 2020.

## Built on by (most cited first, 22 of 22)

- [Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support](https://mlsanity.com/paper/2112.05675/) (2021)
- [How Different Groups Prioritize Ethical Values for Responsible AI](https://mlsanity.com/paper/2205.07722/) (2022)
- [Deconstructing NLG Evaluation: Evaluation Practices, Assumptions, and Their Implications](https://mlsanity.com/paper/2205.06828/) (2022)
- [Understanding Machine Learning Practitioners' Data Documentation Perceptions, Needs, Challenges, and Desiderata](https://mlsanity.com/paper/2206.02923/) (2022)
- [Farsight: Fostering Responsible AI Awareness During AI Application Prototyping](https://mlsanity.com/paper/2402.15350/) (2024)
- [Reality Check: A New Evaluation Ecosystem Is Necessary to Understand AI's Real World Effects](https://mlsanity.com/paper/2505.18893/) (2025)
- [More than Marketing? On the Information Value of AI Benchmarks for Practitioners](https://mlsanity.com/paper/2412.05520/) (2024)
- [AI Ethics Statements -- Analysis and lessons learnt from NeurIPS Broader Impact Statements](https://mlsanity.com/paper/2111.01705/) (2021)
- [Making Intelligence: Ethical Values in IQ and ML Benchmarks](https://mlsanity.com/paper/2209.00692/) (2022)
- [AI Mismatches: Identifying Potential Algorithmic Harms Before AI Development](https://mlsanity.com/paper/2502.18682/) (2025)
- [Human-Centered Responsible Artificial Intelligence: Current & Future Trends](https://mlsanity.com/paper/2302.08157/) (2023)
- [The AI Incident Database as an Educational Tool to Raise Awareness of AI Harms: A Classroom Exploration of Efficacy, Limitations, & Future Improvements](https://mlsanity.com/paper/2310.06269/) (2023)
- [Fairness and representation in satellite-based poverty maps: Evidence of urban-rural disparities and their impacts on downstream policy](https://mlsanity.com/paper/2305.01783/) (2023)
- [Rigor in AI: Doing Rigorous AI Work Requires a Broader, Responsible AI-Informed Conception of Rigor](https://mlsanity.com/paper/2506.14652/) (2025)
- [A Systematic Literature Review of Human-Centered, Ethical, and Responsible AI](https://mlsanity.com/paper/2302.05284/) (2023)
- [Can we Debias Social Stereotypes in AI-Generated Images? Examining Text-to-Image Outputs and User Perceptions](https://mlsanity.com/paper/2505.20692/) (2025)
- [Is a Seat at the Table Enough? Engaging Teachers and Students in Dataset Specification for ML in Education](https://mlsanity.com/paper/2311.05792/) (2023)
- [Legacy Procurement Practices Shape How U.S. Cities Govern AI: Understanding Government Employees' Practices, Challenges, and Needs](https://mlsanity.com/paper/2411.04994/) (2024)
- [Does AI Coaching Prepare us for Workplace Negotiations?](https://mlsanity.com/paper/2509.22545/) (2025)
- [Pragmatic auditing: a pilot-driven approach for auditing Machine Learning systems](https://mlsanity.com/paper/2405.13191/) (2024)
- [Real-World AI Evaluation: How FRAME Generates Systematic Evidence to Resolve the Decision-Maker's Dilemma](https://mlsanity.com/paper/2603.13294/) (2026)
- [Sima AIunty: Caste Audit in LLM-Driven Matchmaking](https://mlsanity.com/paper/2603.29288/) (2026)

## Abstract (arXiv)

NeurIPS 2020 requested that research paper submissions include impact statements on "potential nefarious uses and the consequences of failure." However, as researchers, practitioners and system designers, a key challenge to anticipating risks is overcoming what Clarke (1962) called 'failures of imagination.' The growing research on bias, fairness, and transparency in computational systems aims to illuminate and mitigate harms, and could thus help inform reflections on possible negative impacts of particular pieces of technical work. The prevalent notion of computational harms -- narrowly construed as either allocational or representational harms -- does not fully capture the open, context dependent, and unobservable nature of harms across the wide range of AI infused systems.The current literature focuses on a small range of examples of harms to motivate algorithmic fixes, overlooking the wider scope of probable harms and the way these harms might affect different stakeholders. The system affordances may also exacerbate harms in unpredictable ways, as they determine stakeholders' control(including of non-users) over how they use and interact with a system output. To effectively assist in anticipating harmful uses, we argue that frameworks of harms must be context-aware and consider a wider range of potential stakeholders, system affordances, as well as viable proxies for assessing harms in the widest sense.