# Identification of Systematic Errors of Image Classifiers on Rare Subgroups

arXiv: https://arxiv.org/abs/2303.05072 · HTML: https://mlsanity.com/paper/2303.05072/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 58 references that are in the map.
- First posted on arXiv 9 Mar 2023.

## Builds on (most cited first, 30 of 39)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [Red Teaming Language Models with Language Models](https://mlsanity.com/paper/2202.03286/) (2022)
- [DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2211.01095/) (2022)
- [Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned](https://mlsanity.com/paper/2209.07858/) (2022)
- [Image Segmentation Using Deep Learning: A Survey](https://mlsanity.com/paper/2001.05566/) (2020)
- [Recognition in Terra Incognita](https://mlsanity.com/paper/1807.04975/) (2018)
- [Muse: Text-To-Image Generation via Masked Generative Transformers](https://mlsanity.com/paper/2301.00704/) (2023)
- [A Survey of Deep Learning Techniques for Autonomous Driving](https://mlsanity.com/paper/1910.07738/) (2019)
- [Deep Learning for Generic Object Detection: A Survey](https://mlsanity.com/paper/1809.02165/) (2018)
- [Noise or Signal: The Role of Image Backgrounds in Object Recognition](https://mlsanity.com/paper/2006.09994/) (2020)
- [Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale](https://mlsanity.com/paper/2211.03759/) (2022)
- [Multiaccuracy: Black-Box Post-Processing for Fairness in Classification](https://mlsanity.com/paper/1805.12317/) (2018)
- [No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problems](https://mlsanity.com/paper/2011.12945/) (2020)
- [Domino: Discovering Systematic Errors with Cross-Modal Embeddings](https://mlsanity.com/paper/2203.14960/) (2022)
- [Distilling Model Failures as Directions in Latent Space](https://mlsanity.com/paper/2206.14754/) (2022)

## Built on by (most cited first, 17 of 17)

- [Mass-Producing Failures of Multimodal Systems with Language Models](https://mlsanity.com/paper/2306.12105/) (2023)
- [ImageNet-D: Benchmarking Neural Network Robustness on Diffusion Synthetic Object](https://mlsanity.com/paper/2403.18775/) (2024)
- [DASH: Detection and Assessment of Systematic Hallucinations of VLMs](https://mlsanity.com/paper/2503.23573/) (2025)
- [DiG-IN: Diffusion Guidance for Investigating Networks -- Uncovering Classifier Differences Neuron Visualisations and Visual Counterfactual Explanations](https://mlsanity.com/paper/2311.17833/) (2023)
- [HiBug2: Efficient and Interpretable Error Slice Discovery for Comprehensive Model Debugging](https://mlsanity.com/paper/2501.16751/) (2025)
- [Deep Learning Safety Concerns in Automated Driving Perception](https://mlsanity.com/paper/2309.03774/) (2023)
- [What could go wrong? Discovering and describing failure modes in computer vision](https://mlsanity.com/paper/2408.04471/) (2024)
- [Identifying Systematic Errors in Object Detectors with the SCROD Pipeline](https://mlsanity.com/paper/2309.13489/) (2023)
- [Zero-Shot Distillation for Image Encoders: How to Make Effective Use of Synthetic Data](https://mlsanity.com/paper/2404.16637/) (2024)
- [SynDiff-AD: Improving Semantic Segmentation and End-to-End Autonomous Driving with Synthetic Data from Latent Diffusion Models](https://mlsanity.com/paper/2411.16776/) (2024)
- [CNS-Bench: Benchmarking Image Classifier Robustness Under Continuous Nuisance Shifts](https://mlsanity.com/paper/2507.17651/) (2025)
- [Position: Explainability Research Must Prioritize Foundations over Ad-hoc Methods](https://mlsanity.com/paper/2607.14123/) (2026)
- [Outline of an Independent Systematic Blackbox Test for ML-based Systems](https://mlsanity.com/paper/2401.17062/) (2024)
- [Are Pose Estimators Ready for the Open World? STAGE: A GenAI Toolkit for Auditing 3D Human Pose Estimators](https://mlsanity.com/paper/2408.16536/) (2024)
- [Detecting Systematic Weaknesses in Vision Models along Predefined Human-Understandable Dimensions](https://mlsanity.com/paper/2502.12360/) (2025)
- [Identification of Fine-grained Systematic Errors via Controlled Scene Generation](https://mlsanity.com/paper/2404.07045/) (2024)
- [Efficient Exploration of Image Classifier Failures with Bayesian Optimization and Text-to-Image Models](https://mlsanity.com/paper/2405.02332/) (2024)

## Abstract (arXiv)

Despite excellent average-case performance of many image classifiers, their performance can substantially deteriorate on semantically coherent subgroups of the data that were under-represented in the training data. These systematic errors can impact both fairness for demographic minority groups as well as robustness and safety under domain shift. A major challenge is to identify such subgroups with subpar performance when the subgroups are not annotated and their occurrence is very rare. We leverage recent advances in text-to-image models and search in the space of textual descriptions of subgroups ("prompts") for subgroups where the target model has low performance on the prompt-conditioned synthesized data. To tackle the exponentially growing number of subgroups, we employ combinatorial testing. We denote this procedure as PromptAttack as it can be interpreted as an adversarial attack in a prompt space. We study subgroup coverage and identifiability with PromptAttack in a controlled setting and find that it identifies systematic errors with high accuracy. Thereupon, we apply PromptAttack to ImageNet classifiers and identify novel systematic errors on rare subgroups.