# Testing DNN Image Classifiers for Confusion & Bias Errors

arXiv: https://arxiv.org/abs/1905.07831 · HTML: https://mlsanity.com/paper/1905.07831/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 May 2019.

## Built on by (most cited first, 18 of 18)

- [Emotionally Numb or Empathetic? Evaluating How LLMs Feel Using EmotionBench](https://mlsanity.com/paper/2308.03656/) (2023)
- [How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review](https://mlsanity.com/paper/2107.12045/) (2021)
- [ModelDiff: Testing-Based DNN Similarity Comparison for Model Reuse Detection](https://mlsanity.com/paper/2106.08890/) (2021)
- [Preemptively Pruning Clever-Hans Strategies in Deep Neural Networks](https://mlsanity.com/paper/2304.05727/) (2023)
- [Estimating and Maximizing Mutual Information for Knowledge Distillation](https://mlsanity.com/paper/2110.15946/) (2021)
- [Specifying and Testing k-Safety Properties for Machine-Learning Models](https://mlsanity.com/paper/2206.06054/) (2022)
- [Validating Multimedia Content Moderation Software via Semantic Fusion](https://mlsanity.com/paper/2305.13623/) (2023)
- [Variation of Gender Biases in Visual Recognition Models Before and After Finetuning](https://mlsanity.com/paper/2303.07615/) (2023)
- [RAI4IoE: Responsible AI for Enabling the Internet of Energy](https://mlsanity.com/paper/2309.11691/) (2023)
- [FAST: Boosting Uncertainty-based Test Prioritization Methods for Neural Networks via Feature Selection](https://mlsanity.com/paper/2409.09130/) (2024)
- [GGT: Graph-Guided Testing for Adversarial Sample Detection of Deep Neural Network](https://mlsanity.com/paper/2107.07043/) (2021)
- [DREAM: Debugging and Repairing AutoML Pipelines](https://mlsanity.com/paper/2401.00379/) (2023)
- [Classification, Slippage, Failure and Discovery](https://mlsanity.com/paper/2104.03886/) (2021)
- [Repairing Group-Level Errors for DNNs Using Weighted Regularization](https://mlsanity.com/paper/2203.13612/) (2022)
- [Reformulating van Rijsbergen's Fβ metric for weighted binary cross-entropy](https://mlsanity.com/paper/2210.16458/) (2022)
- [Distribution-aware Fairness Test Generation](https://mlsanity.com/paper/2305.13935/) (2023)
- [Automated Harmfulness Testing for Code Large Language Models](https://mlsanity.com/paper/2503.16740/) (2025)
- [Ethics Testing: Proactive Identification of Generative AI System Harms](https://mlsanity.com/paper/2604.22089/) (2026)

## Abstract (arXiv)

Image classifiers are an important component of today's software, from consumer and business applications to safety-critical domains. The advent of Deep Neural Networks (DNNs) is the key catalyst behind such wide-spread success. However, wide adoption comes with serious concerns about the robustness of software systems dependent on DNNs for image classification, as several severe erroneous behaviors have been reported under sensitive and critical circumstances. We argue that developers need to rigorously test their software's image classifiers and delay deployment until acceptable. We present an approach to testing image classifier robustness based on class property violations. We found that many of the reported erroneous cases in popular DNN image classifiers occur because the trained models confuse one class with another or show biases towards some classes over others. These bugs usually violate some class properties of one or more of those classes. Most DNN testing techniques focus on per-image violations, so fail to detect class-level confusions or biases. We developed a testing technique to automatically detect class-based confusion and bias errors in DNN-driven image classification software. We evaluated our implementation, DeepInspect, on several popular image classifiers with precision up to 100% (avg.~72.6%) for confusion errors, and up to 84.3% (avg.~66.8%) for bias errors. DeepInspect found hundreds of classification mistakes in widely-used models, many exposing errors indicating confusion or bias.