# fAIlureNotes: Supporting Designers in Understanding the Limits of AI Models for Computer Vision Tasks

arXiv: https://arxiv.org/abs/2302.11703 · HTML: https://mlsanity.com/paper/2302.11703/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 93 references that are in the map.
- First posted on arXiv 22 Feb 2023.

## Builds on (most cited first, 30 of 37)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey](https://mlsanity.com/paper/1801.00553/) (2018)
- [Dynabench: Rethinking Benchmarking in NLP](https://mlsanity.com/paper/2104.14337/) (2021)
- [Snorkel: Rapid Training Data Creation with Weak Supervision](https://mlsanity.com/paper/1711.10160/) (2017)
- [Adversarial Attacks on Neural Network Policies](https://mlsanity.com/paper/1702.02284/) (2017)
- [Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imaging](https://mlsanity.com/paper/1909.12475/) (2019)
- [The What-If Tool: Interactive Probing of Machine Learning Models](https://mlsanity.com/paper/1907.04135/) (2019)
- [Gradio: Hassle-Free Sharing and Testing of ML Models in the Wild](https://mlsanity.com/paper/1906.02569/) (2019)
- [Domino: Discovering Systematic Errors with Cross-Modal Embeddings](https://mlsanity.com/paper/2203.14960/) (2022)
- [End-to-end people detection in crowded scenes](https://mlsanity.com/paper/1506.04878/) (2015)
- [The Ethics of AI Ethics -- An Evaluation of Guidelines](https://mlsanity.com/paper/1903.03425/) (2019)
- [Artificial Intelligence: the global landscape of ethics guidelines](https://mlsanity.com/paper/1906.11668/) (2019)
- [Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database](https://mlsanity.com/paper/2011.08512/) (2020)
- [Image Data Augmentation for Deep Learning: A Survey](https://mlsanity.com/paper/2204.08610/) (2022)
- [Automated Data Slicing for Model Validation:A Big data - AI Integration Approach](https://mlsanity.com/paper/1807.06068/) (2018)
- [Interactive Model Cards: A Human-Centered Approach to Model Documentation](https://mlsanity.com/paper/2205.02894/) (2022)
- [Towards Accountable AI: Hybrid Human-Machine Analyses for Characterizing System Failure](https://mlsanity.com/paper/1809.07424/) (2018)
- [Designing Disaggregated Evaluations of AI Systems: Choices, Considerations, and Tradeoffs](https://mlsanity.com/paper/2103.06076/) (2021)
- [FactSheets: Increasing Trust in AI Services through Supplier's Declarations of Conformity](https://mlsanity.com/paper/1808.07261/) (2018)
- [Discovering and Validating AI Errors With Crowdsourced Failure Reports](https://mlsanity.com/paper/2109.11690/) (2021)

## Built on by (most cited first, 10 of 10)

- [Investigating Practices and Opportunities for Cross-functional Collaboration around AI Fairness in Industry Practice](https://mlsanity.com/paper/2306.06542/) (2023)
- [Where Does My Model Underperform? A Human Evaluation of Slice Discovery Algorithms](https://mlsanity.com/paper/2306.08167/) (2023)
- [PromptInfuser: How Tightly Coupling AI and UI Design Impacts Designers' Workflows](https://mlsanity.com/paper/2310.15435/) (2023)
- [Why is AI not a Panacea for Data Workers? An Interview Study on Human-AI Collaboration in Data Storytelling](https://mlsanity.com/paper/2304.08366/) (2023)
- [Policy Maps: Tools for Guiding the Unbounded Space of LLM Behaviors](https://mlsanity.com/paper/2409.18203/) (2024)
- [Situated, Dynamic, and Subjective: Envisioning the Design of Theory-of-Mind-Enabled Everyday AI with Industry Practitioners](https://mlsanity.com/paper/2602.11342/) (2026)
- [Human-Centered AI Product Prototyping with No-Code AutoML: Conceptual Framework, Potentials and Limitations](https://mlsanity.com/paper/2402.07933/) (2024)
- [Documentation Practices of Artificial Intelligence](https://mlsanity.com/paper/2406.18620/) (2024)
- [Exploring the Innovation Opportunities for Pre-trained Models](https://mlsanity.com/paper/2505.15790/) (2025)
- ["I Just Need GPT to Refine My Prompts": Rethinking Onboarding and Help-Seeking with Generative 3D Modeling Tools](https://mlsanity.com/paper/2603.29118/) (2026)

## Abstract (arXiv)

To design with AI models, user experience (UX) designers must assess the fit between the model and user needs. Based on user research, they need to contextualize the model's behavior and potential failures within their product-specific data instances and user scenarios. However, our formative interviews with ten UX professionals revealed that such a proactive discovery of model limitations is challenging and time-intensive. Furthermore, designers often lack technical knowledge of AI and accessible exploration tools, which challenges their understanding of model capabilities and limitations. In this work, we introduced a failure-driven design approach to AI, a workflow that encourages designers to explore model behavior and failure patterns early in the design process. The implementation of fAIlureNotes, a designer-centered failure exploration and analysis tool, supports designers in evaluating models and identifying failures across diverse user groups and scenarios. Our evaluation with UX practitioners shows that fAIlureNotes outperforms today's interactive model cards in assessing context-specific model performance.