# Towards a More Rigorous Science of Blindspot Discovery in Image Classification Models

arXiv: https://arxiv.org/abs/2207.04104 · HTML: https://mlsanity.com/paper/2207.04104/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 71 references that are in the map.
- First posted on arXiv 8 Jul 2022.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [Towards Robust Interpretability with Self-Explaining Neural Networks](https://mlsanity.com/paper/1806.07538/) (2018)
- [What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation](https://mlsanity.com/paper/2008.03703/) (2020)
- [Noise or Signal: The Role of Image Backgrounds in Object Recognition](https://mlsanity.com/paper/2006.09994/) (2020)
- [On Completeness-aware Concept-Based Explanations in Deep Neural Networks](https://mlsanity.com/paper/1910.07969/) (2019)
- [Multiaccuracy: Black-Box Post-Processing for Fairness in Classification](https://mlsanity.com/paper/1805.12317/) (2018)
- [Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imaging](https://mlsanity.com/paper/1909.12475/) (2019)
- [No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problems](https://mlsanity.com/paper/2011.12945/) (2020)
- [Domino: Discovering Systematic Errors with Cross-Modal Embeddings](https://mlsanity.com/paper/2203.14960/) (2022)
- [What shapes feature representations? Exploring datasets, architectures, and training](https://mlsanity.com/paper/2006.12433/) (2020)
- [Explaining in Style: Training a GAN to explain a classifier in StyleSpace](https://mlsanity.com/paper/2104.13369/) (2021)
- [FairVis: Visual Analytics for Discovering Intersectional Bias in Machine Learning](https://mlsanity.com/paper/1904.05419/) (2019)
- [Don't Judge an Object by Its Context: Learning to Overcome Contextual Bias](https://mlsanity.com/paper/2001.03152/) (2020)
- [Leveraging Sparse Linear Layers for Debuggable Deep Networks](https://mlsanity.com/paper/2105.04857/) (2021)
- [Explanation by Progressive Exaggeration](https://mlsanity.com/paper/1911.00483/) (2019)
- [Summit: Scaling Deep Learning Interpretability by Visualizing Activation and Attribution Summarizations](https://mlsanity.com/paper/1904.02323/) (2019)
- [Automated Data Slicing for Model Validation:A Big data - AI Integration Approach](https://mlsanity.com/paper/1807.06068/) (2018)
- [The Spotlight: A General Method for Discovering Systematic Errors in Deep Learning Models](https://mlsanity.com/paper/2107.00758/) (2021)
- [Understanding Failures of Deep Networks via Robust Feature Extraction](https://mlsanity.com/paper/2012.01750/) (2020)
- [Not Using the Car to See the Sidewalk: Quantifying and Controlling the Effects of Context in Classification and Segmentation](https://mlsanity.com/paper/1812.06707/) (2018)
- [Discover the Unknown Biased Attribute of an Image Classifier](https://mlsanity.com/paper/2104.14556/) (2021)
- [Finding and Fixing Spurious Patterns with Explanations](https://mlsanity.com/paper/2106.02112/) (2021)
- [3DB: A Framework for Debugging Computer Vision Models](https://mlsanity.com/paper/2106.03805/) (2021)
- [Sanity Simulations for Saliency Methods](https://mlsanity.com/paper/2105.06506/) (2021)
- [Detecting Spurious Correlations with Sanity Tests for Artificial Intelligence Guided Radiology Systems](https://mlsanity.com/paper/2103.03048/) (2021)
- [DiagViB-6: A Diagnostic Benchmark Suite for Vision Models in the Presence of Shortcut and Generalization Opportunities](https://mlsanity.com/paper/2108.05779/) (2021)

## Built on by (most cited first, 5 of 5)

- [Slicing Through Bias: Explaining Performance Gaps in Medical Image Analysis using Slice Discovery Methods](https://mlsanity.com/paper/2406.12142/) (2024)
- [A hierarchical decomposition for explaining ML performance discrepancies](https://mlsanity.com/paper/2402.14254/) (2024)
- [Finding Patterns in Ambiguity: Interpretable Stress Testing in the Decision~Boundary](https://mlsanity.com/paper/2408.06302/) (2024)
- [Detecting Systematic Weaknesses in Vision Models along Predefined Human-Understandable Dimensions](https://mlsanity.com/paper/2502.12360/) (2025)
- [Fairness Beyond Demographics: Optimizing Performance Across Appearance-Based Hidden Cohorts in Medical Imaging](https://mlsanity.com/paper/2605.29827/) (2026)

## Abstract (arXiv)

A growing body of work studies Blindspot Discovery Methods ("BDM"s): methods that use an image embedding to find semantically meaningful (i.e., united by a human-understandable concept) subsets of the data where an image classifier performs significantly worse. Motivated by observed gaps in prior work, we introduce a new framework for evaluating BDMs, SpotCheck, that uses synthetic image datasets to train models with known blindspots and a new BDM, PlaneSpot, that uses a 2D image representation. We use SpotCheck to run controlled experiments that identify factors that influence BDM performance (e.g., the number of blindspots in a model, or features used to define the blindspot) and show that PlaneSpot is competitive with and in many cases outperforms existing BDMs. Importantly, we validate these findings by designing additional experiments that use real image data from MS-COCO, a large image benchmark dataset. Our findings suggest several promising directions for future work on BDM design and evaluation. Overall, we hope that the methodology and analyses presented in this work will help facilitate a more rigorous science of blindspot discovery.