mlsanityData to 7 Oct 2026

Noise or Signal: The Role of Image Backgrounds in Object Recognition

arXiv 2006.09994 ↗First posted 17 Jun 2020cs.CV cs.LGcited by 342

Not on the map. Why?

Abstract (arXiv)

We assess the tendency of state-of-the-art object recognition models to depend on signals from image backgrounds. We create a toolkit for disentangling foreground and background signal on ImageNet images, and find that (a) models can achieve non-trivial accuracy by relying on the background alone, (b) models often misclassify images even in the presence of correctly classified foregrounds--up to 87.5% of the time with adversarially chosen backgrounds, and (c) more accurate models tend to depend on backgrounds less. Our analysis of backgrounds brings us closer to understanding which correlations machine learning models use, and how they determine models' out of distribution performance.

Key facts. Cited by 342 papers in the map (data through 7 Oct 2026). First posted on arXiv 17 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 49; 2022: 74; 2023: 71; 2024: 60; 2025: 52; 2026: 36.

342 papers.

Sort:

By area:

  1. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
  2. 2021SWAD: Domain Generalization by Seeking Flat Minimacited by 431
  3. 2022Last Layer Re-Training is Sufficient for Robustness to Spurious Correlationscited by 379
  4. 2021Rethinking Spatial Dimensions of Vision Transformerscited by 361
  5. 2022Delving into Out-of-Distribution Detection with Vision-Language Representationscited by 215
  6. 2021Vision Transformers are Robust Learnerscited by 208
  7. 2021Projected GANs Converge Fastercited by 204
  8. 2021A Fine-Grained Analysis on Distribution Shiftcited by 171
  9. 2022On Feature Learning in the Presence of Spurious Correlationscited by 145
  10. 2022Natural Language Descriptions of Deep Visual Featurescited by 136
  11. 2022Domain Generalization by Mutual-Information Regularization with Pre-trained Modelscited by 133
  12. 2023Change is Hard: A Closer Look at Subpopulation Shiftcited by 119
38 more
  1. 2021Re-labeling ImageNet: from Single to Multi-Labels, from Global to Localized Labelscited by 117
  2. 2022What to Hide from Your Students: Attention-Guided Masked Image Modelingcited by 117
  3. 2021Salient ImageNet: How to discover spurious features in Deep Learning?cited by 114
  4. 2021Counterfactual Generative Networkscited by 111
  5. 2021Model-Based Domain Generalizationcited by 103
  6. 2021Long-Short Transformer: Efficient Transformers for Language and Visioncited by 102
  7. 2021Delving Deep into the Generalization of Vision Transformers under Distribution Shiftscited by 85
  8. 2022A Whac-A-Mole Dilemma: Shortcuts Come in Multiples Where Mitigating One Amplifies Otherscited by 85
  9. 2021Evading the Simplicity Bias: Training a Diverse Set of Models Discovers Solutions with Superior OOD Generalizationcited by 83
  10. 2021Leveraging Sparse Linear Layers for Debuggable Deep Networkscited by 82
  11. 2022Distilling Model Failures as Directions in Latent Spacecited by 82
  12. 2022Perceptual Grouping in Contrastive Vision-Language Modelscited by 82
  13. 2022Adversarial Masking for Self-Supervised Learningcited by 78
  14. 2022Generating High Fidelity Data from Low-density Regions using Diffusion Modelscited by 78
  15. 2021Label-Imbalanced and Group-Sensitive Classification under Overparameterizationcited by 74
  16. 2021IAIA-BL: A Case-based Interpretable Deep Learning Model for Classification of Mass Lesions in Digital Mammographycited by 72
  17. 2022Domain-Adjusted Regression or: ERM May Already Learn Features Sufficient for Out-of-Distribution Generalizationcited by 69
  18. 2021On the Impact of Spurious Correlation for Out-of-distribution Detectioncited by 68
  19. 2023Simple and Fast Group Robustness by Automatic Feature Reweightingcited by 68
  20. 2022MaskTune: Mitigating Spurious Correlations by Forcing to Explorecited by 66
  21. 2021Editing a classifier by rewriting its prediction rulescited by 64
  22. 2023On the Foundations of Shortcut Learningcited by 64
  23. 2023Text-To-Concept (and Back) via Cross-Model Alignmentcited by 63
  24. 2023Not All Neuro-Symbolic Concepts Are Created Equal: Analysis and Mitigation of Reasoning Shortcutscited by 63
  25. 2024A Multimodal Automated Interpretability Agentcited by 62
  26. 2022When does dough become a bagel? Analyzing the remaining mistakes on ImageNetcited by 60
  27. 2022Post hoc Explanations may be Ineffective for Detecting Unknown Spurious Correlationcited by 59
  28. 2023Discovering and Mitigating Visual Biases through Keyword Explanationcited by 58
  29. 2022A Comprehensive Study of Image Classification Model Sensitivity to Foregrounds, Backgrounds, and Visual Attributescited by 56
  30. 2023Towards Last-layer Retraining for Group Robustness with Fewer Annotationscited by 54
  31. 2023SocialCounterfactuals: Probing and Mitigating Intersectional Social Biases in Vision-Language Models with Counterfactual Examplescited by 53
  32. 2022Concept-level Debugging of Part-Prototype Networkscited by 52
  33. 2021Rectifying the Shortcut Learning of Background for Few-Shot Learningcited by 51
  34. 2022Mechanistic Mode Connectivitycited by 50
  35. 2022Beyond Attentive Tokens: Incorporating Token Importance and Diversity for Efficient Vision Transformerscited by 47
  36. 2022Discovering Bugs in Vision Models using Off-the-shelf Image Generation and Captioningcited by 45
  37. 2022Spurious Features Everywhere -- Large-Scale Detection of Harmful Spurious Features in ImageNetcited by 41
  38. 2023Spawrious: A Benchmark for Fine Control of Spurious Correlation Biasescited by 41

50 most cited of 342 All 342 (JSON)