# Overcoming Simplicity Bias in Deep Networks using a Feature Sieve

arXiv: https://arxiv.org/abs/2301.13293 · HTML: https://mlsanity.com/paper/2301.13293/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 65 references that are in the map.
- First posted on arXiv 30 Jan 2023.

## Builds on (most cited first, 30 of 38)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Natural Adversarial Examples](https://mlsanity.com/paper/1907.07174/) (2019)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Domain Generalization by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1903.06864/) (2019)
- [Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment](https://mlsanity.com/paper/1610.08452/) (2016)
- [The Pitfalls of Simplicity Bias in Neural Networks](https://mlsanity.com/paper/2006.07710/) (2020)
- [Noise or Signal: The Role of Image Backgrounds in Object Recognition](https://mlsanity.com/paper/2006.09994/) (2020)
- [Environment Inference for Invariant Learning](https://mlsanity.com/paper/2010.07249/) (2020)
- [Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases](https://mlsanity.com/paper/1909.03683/) (2019)
- [Gradient Starvation: A Learning Proclivity in Neural Networks](https://mlsanity.com/paper/2011.09468/) (2020)
- [Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet](https://mlsanity.com/paper/1904.00760/) (2019)
- [Learning Not to Learn: Training Deep Neural Networks with Biased Data](https://mlsanity.com/paper/1812.10352/) (2018)
- [Learning De-biased Representations with Biased Representations](https://mlsanity.com/paper/1910.02806/) (2019)
- [RUBi: Reducing Unimodal Biases in Visual Question Answering](https://mlsanity.com/paper/1906.10169/) (2019)
- [Domain Generalization by Marginal Transfer Learning](https://mlsanity.com/paper/1711.07910/) (2017)
- [REPAIR: Removing Representation Bias by Dataset Resampling](https://mlsanity.com/paper/1904.07911/) (2019)
- [Understanding the Failure Modes of Out-of-Distribution Generalization](https://mlsanity.com/paper/2010.15775/) (2020)
- [Learning from Failure: Training Debiased Classifier from Biased Classifier](https://mlsanity.com/paper/2007.02561/) (2020)
- [What shapes feature representations? Exploring datasets, architectures, and training](https://mlsanity.com/paper/2006.12433/) (2020)
- [EnD: Entangling and Disentangling deep representations for bias correction](https://mlsanity.com/paper/2103.02023/) (2021)
- [Evading the Simplicity Bias: Training a Diverse Set of Models Discovers Solutions with Superior OOD Generalization](https://mlsanity.com/paper/2105.05612/) (2021)
- [Causal Attention for Unbiased Visual Recognition](https://mlsanity.com/paper/2108.08782/) (2021)
- [BiaSwap: Removing dataset bias with bias-tailored swapping augmentation](https://mlsanity.com/paper/2108.10008/) (2021)

## Built on by (most cited first, 29 of 29)

- [On the Foundations of Shortcut Learning](https://mlsanity.com/paper/2310.16228/) (2023)
- [The Clever Hans Mirage: A Comprehensive Survey on Spurious Correlations in Machine Learning](https://mlsanity.com/paper/2402.12715/) (2024)
- [Spurious Correlations in High Dimensional Regression: The Roles of Regularization, Simplicity Bias and Over-Parameterization](https://mlsanity.com/paper/2502.01347/) (2025)
- [Using Early Readouts to Mediate Featural Bias in Distillation](https://mlsanity.com/paper/2310.18590/) (2023)
- [Spuriousness-Aware Meta-Learning for Learning Robust Classifiers](https://mlsanity.com/paper/2406.10742/) (2024)
- [Severing Spurious Correlations with Data Pruning](https://mlsanity.com/paper/2503.18258/) (2025)
- [DeNetDM: Debiasing by Network Depth Modulation](https://mlsanity.com/paper/2403.19863/) (2024)
- [Efficient Unsupervised Shortcut Learning Detection and Mitigation in Transformers](https://mlsanity.com/paper/2501.00942/) (2025)
- [CHARM3R: Towards Unseen Camera Height Robust Monocular 3D Detector](https://mlsanity.com/paper/2508.11185/) (2025)
- [Adaptive Group Robust Ensemble Knowledge Distillation](https://mlsanity.com/paper/2411.14984/) (2024)
- [ShortcutProbe: Probing Prediction Shortcuts for Learning Robust Models](https://mlsanity.com/paper/2505.13910/) (2025)
- [Sebra: Debiasing Through Self-Guided Bias Ranking](https://mlsanity.com/paper/2501.18277/) (2025)
- [Changing the Training Data Distribution to Reduce Simplicity Bias Improves In-distribution Generalization](https://mlsanity.com/paper/2404.17768/) (2024)
- [Looking at Model Debiasing through the Lens of Anomaly Detection](https://mlsanity.com/paper/2407.17449/) (2024)
- [UnLearning from Experience to Avoid Spurious Correlations](https://mlsanity.com/paper/2409.02792/) (2024)
- [NeuronTune: Towards Self-Guided Spurious Bias Mitigation](https://mlsanity.com/paper/2505.24048/) (2025)
- [Large Learning Rates Simultaneously Achieve Robustness to Spurious Correlations and Compressibility](https://mlsanity.com/paper/2507.17748/) (2025)
- [Concept Extrapolation: A Conceptual Primer](https://mlsanity.com/paper/2306.10999/) (2023)
- [Enhancing Self-Supervised Visual Representation Learning via Low-Rank Adapted LLMs](https://mlsanity.com/paper/2507.00754/) (2025)
- [Let Samples Speak: Mitigating Spurious Correlation by Exploiting the Clusterness of Samples](https://mlsanity.com/paper/2512.22874/) (2025)
- [Mitigating Simplicity Bias in OOD Detection through Object Co-occurrence Analysis](https://mlsanity.com/paper/2605.07821/) (2026)
- [Forget Sharpness: Perturbed Forgetting of Model Biases Within SAM Dynamics](https://mlsanity.com/paper/2406.06700/) (2024)
- [Debiasify: Self-Distillation for Unsupervised Bias Mitigation](https://mlsanity.com/paper/2411.00711/) (2024)
- [A Critical Review of Predominant Bias in Neural Networks](https://mlsanity.com/paper/2502.11031/) (2025)
- [Divide and Conquer Self-Supervised Learning for High-Content Imaging](https://mlsanity.com/paper/2503.07444/) (2025)
- [Token-Based Detection of Spurious Correlations in Vision Transformers](https://mlsanity.com/paper/2509.04009/) (2025)
- [ACE and Diverse Generalization via Selective Disagreement](https://mlsanity.com/paper/2509.07955/) (2025)
- [Anatomy of a failure: When, how, and why deep vision fails in scientific domains](https://mlsanity.com/paper/2605.04231/) (2026)
- [The Conflict Between Logic and Memory: Training Conditions for Optimizer-Dependent Rule Acquisition](https://mlsanity.com/paper/2610.00403/) (2026)

## Abstract (arXiv)

Simplicity bias is the concerning tendency of deep networks to over-depend on simple, weakly predictive features, to the exclusion of stronger, more complex features. This is exacerbated in real-world applications by limited training data and spurious feature-label correlations, leading to biased, incorrect predictions. We propose a direct, interventional method for addressing simplicity bias in DNNs, which we call the feature sieve. We aim to automatically identify and suppress easily-computable spurious features in lower layers of the network, thereby allowing the higher network levels to extract and utilize richer, more meaningful representations. We provide concrete evidence of this differential suppression & enhancement of relevant features on both controlled datasets and real-world images, and report substantial gains on many real-world debiasing benchmarks (11.4% relative gain on Imagenet-A; 3.2% on BAR, etc). Crucially, we do not depend on prior knowledge of spurious attributes or features, and in fact outperform many baselines that explicitly incorporate such information. We believe that our feature sieve work opens up exciting new research directions in automated adversarial feature extraction and representation learning for deep networks.