mlsanityData to 14 Oct 2026

Discovering Bugs in Vision Models using Off-the-shelf Image Generation and Captioning

arXiv 2208.08831 ↗First posted 18 Aug 2022cs.CV cs.LG stat.MLcited by 45

Not on the map. Why?

Abstract (arXiv)

Automatically discovering failures in vision models under real-world settings remains an open challenge. This work demonstrates how off-the-shelf, large-scale, image-to-text and text-to-image models, trained on vast amounts of data, can be leveraged to automatically find such failures. In essence, a conditional text-to-image generative model is used to generate large amounts of synthetic, yet realistic, inputs given a ground-truth label. Misclassified inputs are clustered and a captioning model is used to describe each cluster. Each cluster's description is used in turn to generate more inputs and assess whether specific clusters induce more failures than expected. We use this pipeline to demonstrate that we can effectively interrogate classifiers trained on ImageNet to find specific failure cases and discover spurious correlations. We also show that we can scale the approach to generate adversarial datasets targeting specific classifier architectures. This work serves as a proof-of-concept demonstrating the utility of large-scale generative models to automatically discover bugs in vision models in an open-ended manner. We also describe a number of limitations and pitfalls related to this approach.

Key facts. Cited by 45 papers in the map (data through 14 Oct 2026). Builds on 56 of its 73 references that are in the map. First posted on arXiv 18 Aug 2022.

Builds on

56 of 73 references.

References · 56

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  6. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  9. 2013Intriguing properties of neural networkscited by 5,155
  10. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  11. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  12. 2014Going Deeper with Convolutionscited by 4,241
38 more
  1. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  2. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  3. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  4. 2013Describing Textures in the Wildcited by 2,125
  5. 2020Shortcut Learning in Deep Neural Networkscited by 1,969
  6. 2019Invariant Risk Minimizationcited by 1,854
  7. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,850
  8. 2016Adversarial examples in the physical worldcited by 1,769
  9. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,515
  10. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,512
  11. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,432
  12. 2019Natural Adversarial Examplescited by 1,408
  13. 2018Demystifying MMD GANscited by 1,293
  14. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,247
  15. 2022Red Teaming Language Models with Language Modelscited by 1,026
  16. 2016End to End Learning for Self-Driving Carscited by 1,009
  17. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 1,006
  18. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 919
  19. 2019Big Transfer (BiT): General Visual Representation Learningcited by 769
  20. 2017Evasion Attacks against Machine Learning at Test Timecited by 689
  21. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 571
  22. 2021How to train your ViT? Data, Augmentation, and Regularization in Vision Transformerscited by 569
  23. 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 546
  24. 2019Unmasking Clever Hans Predictors and Assessing What Machines Really Learncited by 365
  25. 2020Noise or Signal: The Role of Image Backgrounds in Object Recognitioncited by 343
  26. 2018Generating Adversarial Examples with Adversarial Networkscited by 258
  27. 2021Scaling Up Vision-Language Pre-training for Image Captioningcited by 196
  28. 2021Partial success in closing the gap between human and machine visioncited by 185
  29. 2022Domino: Discovering Systematic Errors with Cross-Modal Embeddingscited by 156
  30. 2018Counterfactual Fairness in Text Classification through Robustnesscited by 151
  31. 2020Perceptual Adversarial Robustness: Defense Against Unseen Threat Modelscited by 137
  32. 2018Constructing Unrestricted Adversarial Examples with Generative Modelscited by 135
  33. 2019SemanticAdv: Generating Adversarial Examples via Attribute-conditional Image Editingcited by 93
  34. 2022Distilling Model Failures as Directions in Latent Spacecited by 82
  35. 2020Beyond accuracy: quantifying trial-by-trial behaviour of CNNs and humans by measuring error consistencycited by 78
  36. 2017Adversarial Transformation Networks: Learning to Generate Adversarial Examplescited by 62
  37. 2021Meaningfully Debugging Model Mistakes using Conceptual Counterfactual Explanationscited by 53
  38. 2020Learning perturbation sets for robust machine learningcited by 48

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 17; 2024: 19; 2025: 7; 2026: 1.

45 papers.

Sort:

By area:

  1. 2024An Introduction to Vision-Language Modelingcited by 178
  2. 2023Sociotechnical Safety Evaluation of Generative AI Systemscited by 157
  3. 2023Discovering and Mitigating Visual Biases through Keyword Explanationcited by 58
  4. 2023LANCE: Stress-testing Visual Models by Generating Language-guided Counterfactual Imagescited by 46
  5. 2023The Journey, Not the Destination: How Data Guides Diffusion Modelscited by 43
  6. 2023Spawrious: A Benchmark for Fine Control of Spurious Correlation Biasescited by 41
  7. 2022Adaptive Testing of Computer Vision Modelscited by 39
  8. 2023Mass-Producing Failures of Multimodal Systems with Language Modelscited by 36
  9. 2023Dataset Interfaces: Diagnosing Model Failures Using Controllable Counterfactual Generationcited by 35
  10. 2023Identifying Interpretable Subspaces in Image Representationscited by 32
  11. 2023Effective Human-AI Teams via Learned Natural Language Rules and Onboardingcited by 20
  12. 2023Identification of Systematic Errors of Image Classifiers on Rare Subgroupscited by 17
33 more
  1. 2024Efficient Lifelong Model Evaluation in an Era of Rapid Progresscited by 10
  2. 2023Where Does My Model Underperform? A Human Evaluation of Slice Discovery Algorithmscited by 9
  3. 2023Error Discovery by Clustering Influence Embeddingscited by 8
  4. 2024Clarify: Improving Model Robustness With Natural Language Correctionscited by 8
  5. 2024Distributionally Generative Augmentation for Fair Facial Attribute Classificationcited by 8
  6. 2025SpurLens: Automatic Detection of Spurious Cues in Multimodal LLMscited by 7
  7. 2024Benchmarking Segmentation Models with Mask-Preserved Attribute Editingcited by 6
  8. 2024Visual Data Diagnosis and Debiasing with Concept Graphscited by 6
  9. 2023ASPIRE: Language-Guided Data Augmentation for Improving Robustness Against Spurious Correlationscited by 5
  10. 2024What could go wrong? Discovering and describing failure modes in computer visioncited by 5
  11. 2025VISLIX: An XAI Framework for Validating Vision Models with Slice Discovery and Analysiscited by 5
  12. 2023Identifying Systematic Errors in Object Detectors with the SCROD Pipelinecited by 4
  13. 2024Say My Name: a Model's Bias Discovery Frameworkcited by 4
  14. 2024Learning Where to Edit Vision Transformerscited by 4
  15. 2025Error Slice Discovery via Manifold Compactnesscited by 4
  16. 2025Classifier-to-Bias: Toward Unsupervised Automatic Bias Detection for Visual Classifierscited by 4
  17. 2023Identifying and Mitigating Model Failures through Few-shot CLIP-aided Diffusion Generationcited by 3
  18. 2024GIFT: A Framework Towards Global Interpretable Faithful Textual Explanations of Vision Classifierscited by 2
  19. 2026Benchmarking Semantic Segmentation Models via Appearance and Geometry Attribute Editingcited by 2
  20. 2024Reinforcing Pre-trained Models Using Counterfactual Imagescited by 1
  21. 2024Unveiling Glitches: A Deep Dive into Image Encoding Bugs within CLIPcited by 1
  22. 2024Are Pose Estimators Ready for the Open World? STAGE: A GenAI Toolkit for Auditing 3D Human Pose Estimatorscited by 1
  23. 2024Explaining an image classifier with a generative model conditioned by uncertaintycited by 1
  24. 2024Interactive Visual Assessment for Text-to-Image Generation Modelscited by 1
  25. 2025Detecting Systematic Weaknesses in Vision Models along Predefined Human-Understandable Dimensionscited by 1
  26. 2025Fine-Grained Class-Conditional Distribution Balancing for Debiased Learningcited by 1
  27. 2025CUBIC: Concept Embeddings for Unsupervised Bias Identification using VLMscited by 1
  28. 2023Targeted Image Data Augmentation Increases Basic Skills Captioning Robustnesscited by 0
  29. 2023Probing clustering in neural network representationscited by 0
  30. 2024Identification of Fine-grained Systematic Errors via Controlled Scene Generationcited by 0
  31. 2024Efficient Exploration of Image Classifier Failures with Bayesian Optimization and Text-to-Image Modelscited by 0
  32. 2024Model editing for distribution shifts in uranium oxide morphological analysiscited by 0
  33. 2024Improving Object Detection by Modifying Synthetic Data with Explainable AIcited by 0