mlsanityData to 14 Oct 2026

Discovering and Validating AI Errors With Crowdsourced Failure Reports

arXiv 2109.11690 ↗First posted 23 Sep 2021cs.HC cs.LGcited by 28

Not on the map. Why?

Abstract (arXiv)

AI systems can fail to learn important behaviors, leading to real-world issues like safety concerns and biases. Discovering these systematic failures often requires significant developer attention, from hypothesizing potential edge cases to collecting evidence and validating patterns. To scale and streamline this process, we introduce crowdsourced failure reports, end-user descriptions of how or why a model failed, and show how developers can use them to detect AI errors. We also design and implement Deblinder, a visual analytics system for synthesizing failure reports that developers can use to discover and validate systematic failures. In semi-structured interviews and think-aloud studies with 10 AI practitioners, we explore the affordances of the Deblinder system and the applicability of failure reports in real-world settings. Lastly, we show how collecting additional data from the groups identified by developers can improve model performance.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Builds on 18 of its 70 references that are in the map. First posted on arXiv 23 Sep 2021.

Builds on

18 of 70 references.

References · 18

  1. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  2. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,054
  3. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,515
  4. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,362
  5. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  6. 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,089
  7. 2018The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scalecited by 544
  8. 2018Improving fairness in machine learning systems: What do industry practitioners need?cited by 262
  9. 2016Data Programming: Creating Large Training Sets, Quicklycited by 246
  10. 2019FairVis: Visual Analytics for Discovering Intersectional Bias in Machine Learningcited by 93
  11. 2018Automated Data Slicing for Model Validation:A Big data - AI Integration Approachcited by 70
  12. 2019FairSight: Visual Analytics for Fairness in Decision Makingcited by 59
6 more
  1. 2018Towards Accountable AI: Hybrid Human-Machine Analyses for Characterizing System Failurecited by 49
  2. 2016Identifying Unknown Unknowns in the Open World: Representations and Policies for Guided Explorationcited by 26
  3. 2019Slice-based Learning: A Programming Model for Residual Learning in Critical Data Slicescited by 23
  4. 2020Inspector Gadget: A Data Programming-based Labeling System for Industrial Imagescited by 5
  5. 2018Interactive Classification for Deep Learning Interpretationcited by 2
  6. 2015Designing for Collaborative Sensemaking: Using Expert & Non-Expert Crowdcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 6; 2023: 10; 2024: 7; 2025: 2; 2026: 2.

28 papers.

Sort:

By area:

  1. 2023Increasing Diversity While Maintaining Accuracy: Text Data Generation with Large Language Models and Human Interventionscited by 125
  2. 2023EvalLM: Interactive Evaluation of Large Language Model Prompts on User-Defined Criteriacited by 74
  3. 2023Supporting Human-AI Collaboration in Auditing LLMs with LLMscited by 46
  4. 2023Improving Human-AI Collaboration With Descriptions of AI Behaviorcited by 36
  5. 2022Towards Transparency in Dermatology Image Datasets with Skin Tone Annotations by Experts, Crowds, and an Algorithmcited by 35
  6. 2022Exploring How Machine Learning Practitioners (Try To) Use Fairness Toolkitscited by 34
  7. 2022Understanding Practices, Challenges, and Opportunities for User-Engaged Algorithm Auditing in Industry Practicecited by 32
  8. 2024Wikibench: Community-Driven Data Curation for AI Evaluation on Wikipediacited by 18
  9. 2024Position: Towards Bidirectional Human-AI Alignmentcited by 18
  10. 2023Investigating Practices and Opportunities for Cross-functional Collaboration around AI Fairness in Industry Practicecited by 15
  11. 2023Angler: Helping Machine Translation Practitioners Prioritize Model Improvementscited by 12
  12. 2023fAIlureNotes: Supporting Designers in Understanding the Limits of AI Models for Computer Vision Taskscited by 10
16 more
  1. 2022A.I. Robustness: a Human-Centered Perspective on Technological Challenges and Opportunitiescited by 9
  2. 2023Where Does My Model Underperform? A Human Evaluation of Slice Discovery Algorithmscited by 9
  3. 2024Clarify: Improving Model Robustness With Natural Language Correctionscited by 8
  4. 2024Machine Learning Robustness: A Primercited by 8
  5. 2022Perspectives on Incorporating Expert Feedback into Model Updatescited by 7
  6. 2023A Systematic Literature Review of Human-Centered, Ethical, and Responsible AIcited by 6
  7. 2021In Search of Ambiguity: A Three-Stage Workflow Design to Clarify Annotation Guidelines for Crowd Workerscited by 5
  8. 2025VISLIX: An XAI Framework for Validating Vision Models with Slice Discovery and Analysiscited by 5
  9. 2022Capabilities for Better ML Engineeringcited by 3
  10. 2024Optimizing Delegation in Collaborative Human-AI Hybrid Teamscited by 3
  11. 2025Position: Beyond Assistance -- Reimagining LLMs as Ethical and Adaptive Co-Creators in Mental Health Carecited by 3
  12. 2023ESCAPE: Countering Systematic Errors from Machine's Blind Spots via Interactive Visual Analysiscited by 2
  13. 2024Optimizing Risk-averse Human-AI Hybrid Teamscited by 1
  14. 2024Illuminating Blind Spots of Language Models with Targeted Agent-in-the-Loop Synthetic Datacited by 1
  15. 2026PersonaTeaming: Supporting Persona-Driven Red-Teaming for Generative AIcited by 0
  16. 2026RuleEdit: Failure-Guided Human-AI Model Editing with Prospective Impact Previewcited by 0