mlsanityData to 14 Oct 2026

Detecting Spurious Correlations with Sanity Tests for Artificial Intelligence Guided Radiology Systems

arXiv 2103.03048 ↗First posted 4 Mar 2021eess.IV cs.CV cs.LGcited by 10

Not on the map. Why?

Abstract (arXiv)

Artificial intelligence (AI) has been successful at solving numerous problems in machine perception. In radiology, AI systems are rapidly evolving and show progress in guiding treatment decisions, diagnosing, localizing disease on medical images, and improving radiologists' efficiency. A critical component to deploying AI in radiology is to gain confidence in a developed system's efficacy and safety. The current gold standard approach is to conduct an analytical validation of performance on a generalization dataset from one or more institutions, followed by a clinical validation study of the system's efficacy during deployment. Clinical validation studies are time-consuming, and best practices dictate limited re-use of analytical validation data, so it is ideal to know ahead of time if a system is likely to fail analytical or clinical validation. In this paper, we describe a series of sanity tests to identify when a system performs well on development data for the wrong reasons. We illustrate the sanity tests' value by designing a deep learning system to classify pancreatic cancer seen in computed tomography scans.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 29 of its 89 references that are in the map. First posted on arXiv 4 Mar 2021.

Builds on

29 of 89 references.

References · 29

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2020Shortcut Learning in Deep Neural Networkscited by 1,969
  3. 2019Invariant Risk Minimizationcited by 1,854
  4. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,512
  5. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,432
  6. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  7. 2018Sanity Checks for Saliency Mapscited by 1,156
  8. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 953
  9. 2018Mitigating Unwanted Biases with Adversarial Learningcited by 663
  10. 2017Interpretation of Neural Networks is Fragilecited by 444
  11. 2018Confounding variables can degrade generalization performance of radiological deep learning modelscited by 379
  12. 2019A large annotated medical image dataset for the development and evaluation of segmentation algorithmscited by 373
17 more
  1. 2019Transfusion: Understanding Transfer Learning for Medical Imagingcited by 368
  2. 2019Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biasescited by 280
  3. 2019Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imagingcited by 216
  4. 2019Causality matters in medical imagingcited by 193
  5. 2019RUBi: Reducing Unimodal Biases in Visual Question Answeringcited by 185
  6. 2019Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Groundedcited by 128
  7. 2019"How do I fool you?": Manipulating User Trust via Misleading Black Box Explanationscited by 111
  8. 2019Towards Causal VQA: Revealing and Reducing Spurious Correlations by Invariant and Covariant Semantic Editingcited by 108
  9. 2018Overcoming Language Priors in Visual Question Answering with Adversarial Regularizationcited by 105
  10. 2020Don't Judge an Object by Its Context: Learning to Overcome Contextual Biascited by 91
  11. 2017Measuring the tendency of CNNs to Learn Surface Statistical Regularitiescited by 87
  12. 2019Two Causal Principles for Improving Visual Dialogcited by 75
  13. 2020On the Value of Out-of-Distribution Testing: An Example of Goodhart's Lawcited by 74
  14. 2020Machine-Learning-Based Multiple Abnormality Prediction with Large-Scale Chest Computed Tomography Volumescited by 67
  15. 2020An Empirical Study of Invariant Risk Minimizationcited by 35
  16. 2019Challenges and Prospects in Vision and Language Researchcited by 24
  17. 2020Visual Grounding Methods for VQA are Working for the Wrong Reasons!cited by 18

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 4; 2023: 2; 2024: 1; 2025: 2; 2026: 1.

10 papers.

  1. 2022DC-Check: A Data-Centric AI checklist to guide the development of reliable machine learning systemscited by 14
  2. 2022Adversarial Counterfactual Augmentation: Application in Alzheimer's Disease Classificationcited by 6
  3. 2022Towards a More Rigorous Science of Blindspot Discovery in Image Classification Modelscited by 5
  4. 2023A Quantitatively Interpretable Model for Alzheimer's Disease Prediction Using Deep Counterfactualscited by 1
  5. 2024INSIGHT: Explainable Weakly-Supervised Medical Image Analysiscited by 1
  6. 2025Detecting Systematic Weaknesses in Vision Models along Predefined Human-Understandable Dimensionscited by 1
  7. 2022Explainer Divergence Scores (EDS): Some Post-Hoc Explanations May be Effective for Detecting Unknown Spurious Correlationscited by 0
  8. 2023Anatomically aware dual-hop learning for pulmonary embolism detection in CT pulmonary angiogramscited by 0
  9. 2025Detecting Dataset Bias in Medical AI: A Generalized and Modality-Agnostic Auditing Frameworkcited by 0
  10. 2026Agentic AIs Are the Missing Paradigm for Out-of-Distribution Generalization in Foundation Modelscited by 0