mlsanityData to 7 Oct 2026

The Spotlight: A General Method for Discovering Systematic Errors in Deep Learning Models

arXiv 2107.00758 ↗First posted 1 Jul 2021cs.LG stat.MLcited by 63

Not on the map. Why?

Abstract (arXiv)

Supervised learning models often make systematic errors on rare subsets of the data. When these subsets correspond to explicit labels in the data (e.g., gender, race) such poor performance can be identified straightforwardly. This paper introduces a method for discovering systematic errors that do not correspond to such explicitly labelled subgroups. The key idea is that similar inputs tend to have similar representations in the final hidden layer of a neural network. We leverage this structure by "shining a spotlight" on this representation space to find contiguous regions where the model performs poorly. We show that the spotlight surfaces semantically meaningful areas of weakness in a wide variety of existing models spanning computer vision, NLP, and recommender systems.

Key facts. Cited by 63 papers in the map (data through 7 Oct 2026). Builds on 25 of its 56 references that are in the map. First posted on arXiv 1 Jul 2021.

Builds on

25 of 56 references.

References · 25

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2013Intriguing properties of neural networkscited by 5,114
  3. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  5. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  6. 2019Invariant Risk Minimizationcited by 1,832
  7. 2018Model Cards for Model Reportingcited by 1,068
  8. 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 488
  9. 2018Fairness Without Demographics in Repeated Loss Minimizationcited by 363
  10. 2020Fairness without Demographics through Adversarially Reweighted Learningcited by 251
  11. 2018Multiaccuracy: Black-Box Post-Processing for Fairness in Classificationcited by 221
  12. 2019Visualizing and Measuring the Geometry of BERTcited by 215
13 more
  1. 2019Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imagingcited by 215
  2. 2020No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problemscited by 208
  3. 2019The What-If Tool: Interactive Probing of Machine Learning Modelscited by 207
  4. 2020Minimax Pareto Fairness: A Multi Objective Perspectivecited by 152
  5. 2019Does Object Recognition Work for Everyone?cited by 149
  6. 2018Why Is My Classifier Discriminatory?cited by 143
  7. 2019Distributionally Robust Language Modelingcited by 142
  8. 2019Cycle-Consistency for Robust Visual Question Answeringcited by 105
  9. 2019FairVis: Visual Analytics for Discovering Intersectional Bias in Machine Learningcited by 93
  10. 2018Automated Data Slicing for Model Validation:A Big data - AI Integration Approachcited by 70
  11. 2018Deep Models of Interactions Across Setscited by 68
  12. 2019FairSight: Visual Analytics for Fairness in Decision Makingcited by 59
  13. 2020Distributionally Robust Losses for Latent Covariate Mixturescited by 57

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 10; 2023: 20; 2024: 14; 2025: 9; 2026: 9.

63 papers.

Sort:

By area:

  1. 2022Domino: Discovering Systematic Errors with Cross-Modal Embeddingscited by 155
  2. 2022A Whac-A-Mole Dilemma: Shortcuts Come in Multiples Where Mitigating One Amplifies Otherscited by 85
  3. 2023Discovering and Mitigating Visual Biases through Keyword Explanationcited by 58
  4. 2023Diagnosing and Rectifying Vision Models using Languagecited by 58
  5. 2023Describing Differences in Image Sets with Natural Languagecited by 51
  6. 2022Perfectly Balanced: Improving Transfer and Robustness of Supervised Contrastive Learningcited by 40
  7. 2022Adaptive Testing of Computer Vision Modelscited by 39
  8. 2023Effective Human-AI Teams via Learned Natural Language Rules and Onboardingcited by 20
  9. 2022"Why did the Model Fail?": Attributing Model Performance Changes to Distribution Shiftscited by 18
  10. 2023Identification of Systematic Errors of Image Classifiers on Rare Subgroupscited by 17
  11. 2022SEAL : Interactive Tool for Systematic Error Analysis and Labelingcited by 15
  12. 2023Targeted Data Generation: Finding and Fixing Model Weaknessescited by 15
38 more
  1. 2024A Survey on Safe Multi-Modal Learning Systemcited by 15
  2. 2024Slicing Through Bias: Explaining Performance Gaps in Medical Image Analysis using Slice Discovery Methodscited by 14
  3. 2023Angler: Helping Machine Translation Practitioners Prioritize Model Improvementscited by 11
  4. 2023Bias in Pruned Vision Models: In-Depth Analysis and Countermeasurescited by 11
  5. 2025HiBug2: Efficient and Interpretable Error Slice Discovery for Comprehensive Model Debuggingcited by 11
  6. 2022Identifying Spurious Correlations and Correcting them with an Explanation-based Learningcited by 10
  7. 2022AGRO: Adversarial Discovery of Error-prone groups for Robust Optimizationcited by 10
  8. 2024Efficient Lifelong Model Evaluation in an Era of Rapid Progresscited by 10
  9. 2023Where Does My Model Underperform? A Human Evaluation of Slice Discovery Algorithmscited by 9
  10. 2023PRIME: Prioritizing Interpretability in Failure Mode Extractioncited by 9
  11. 2023Capturing Humans' Mental Models of AI: An Item Response Theory Approachcited by 8
  12. 2024Prompt-driven Latent Domain Generalization for Medical Image Classificationcited by 8
  13. 2024Clarify: Improving Model Robustness With Natural Language Correctionscited by 8
  14. 2024Efficient Bias Mitigation Without Privileged Informationcited by 8
  15. 2024Context-Aware Testing: A New Paradigm for Model Testing with Large Language Modelscited by 8
  16. 2023VLSlice: Interactive Vision-and-Language Slice Discoverycited by 7
  17. 2023Error Discovery by Clustering Influence Embeddingscited by 7
  18. 2026ProbeLLM: Automating Principled Diagnosis of LLM Failurescited by 7
  19. 2024AttributionScanner: A Visual Analytics System for Model Validation with Metadata-Free Slice Findingcited by 6
  20. 2021How Emotionally Stable is ALBERT? Testing Robustness with Stochastic Weight Averaging on a Sentiment Analysis Taskcited by 5
  21. 2022Towards a More Rigorous Science of Blindspot Discovery in Image Classification Modelscited by 5
  22. 2024Learning Exceptional Subgroups by End-to-End Maximizing KL-divergencecited by 5
  23. 2024A hierarchical decomposition for explaining ML performance discrepanciescited by 5
  24. 2024What could go wrong? Discovering and describing failure modes in computer visioncited by 5
  25. 2024Policy Maps: Tools for Guiding the Unbounded Space of LLM Behaviorscited by 5
  26. 2025VISLIX: An XAI Framework for Validating Vision Models with Slice Discovery and Analysiscited by 5
  27. 2022Discover, Explanation, Improvement: An Automatic Slice Detection Framework for Natural Language Processingcited by 4
  28. 2025Classifier-to-Bias: Toward Unsupervised Automatic Bias Detection for Visual Classifierscited by 4
  29. 2023Mitigating Source Bias for Fairer Weak Supervisioncited by 3
  30. 2025Subgroup Performance Analysis in Hidden Stratificationscited by 3
  31. 2023ESCAPE: Countering Systematic Errors from Machine's Blind Spots via Interactive Visual Analysiscited by 2
  32. 2023TLDR: Text Based Last-layer Retraining for Debiasing Image Classifierscited by 2
  33. 2025TRoVe: Discovering Error-Inducing Static Feature Biases in Temporal Vision-Language Modelscited by 2
  34. 2023Assessing Systematic Weaknesses of DNNs using Counterfactualscited by 1
  35. 2023Language-assisted Vision Model Debugger: A Sample-Free Approach to Finding and Fixing Bugscited by 1
  36. 2025Detecting Systematic Weaknesses in Vision Models along Predefined Human-Understandable Dimensionscited by 1
  37. 2025Active Slice Discovery in Large Language Modelscited by 1
  38. 2026Measurement Under Selection: Decoy-Calibrated Failure Audits for Language Modelscited by 1

50 most cited of 63 All 63 (JSON)