mlsanityData to 7 Oct 2026

DeNetDM: Debiasing by Network Depth Modulation

arXiv 2403.19863 ↗First posted 28 Mar 2024cs.LG cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

Neural networks trained on biased datasets tend to inadvertently learn spurious correlations, hindering generalization. We formally prove that (1) samples that exhibit spurious correlations lie on a lower rank manifold relative to the ones that do not; and (2) the depth of a network acts as an implicit regularizer on the rank of the attribute subspace that is encoded in its representations. Leveraging these insights, we present DeNetDM, a novel debiasing method that uses network depth modulation as a way of developing robustness to spurious correlations. Using a training paradigm derived from Product of Experts, we create both biased and debiased branches with deep and shallow architectures and then distill knowledge to produce the target debiased model. Our method requires no bias annotations or explicit data augmentation while performing on par with approaches that require either or both. We demonstrate that DeNetDM outperforms existing debiasing techniques on both synthetic and real-world datasets by 5%. The project page is available at https://vssilpa.github.io/denetdm/.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 36 of its 51 references that are in the map. First posted on arXiv 28 Mar 2024.

Builds on

36 of 51 references.

References · 36

  1. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  2. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  4. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  5. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  6. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  7. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
  8. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,232
  9. 2021Just Train Twice: Improving Group Robustness without Training Group Informationcited by 522
  10. 2020The Pitfalls of Simplicity Bias in Neural Networkscited by 349
  11. 2020Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learningcited by 292
  12. 2019Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biasescited by 279
24 more
  1. 2018Learning Not to Learn: Training Deep Neural Networks with Biased Datacited by 255
  2. 2019Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigationcited by 252
  3. 2020No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problemscited by 208
  4. 2016Deep Information Propagationcited by 194
  5. 2020Invariant Risk Minimization Gamescited by 188
  6. 2021The Low-Rank Simplicity Bias in Deep Networkscited by 150
  7. 2019End-to-End Bias Mitigation by Modelling Biases in Corporacited by 127
  8. 2020What shapes feature representations? Exploring datasets, architectures, and trainingcited by 125
  9. 2019Learning Robust Representations by Projecting Superficial Statistics Outcited by 123
  10. 2021Learning Debiased Representation via Disentangled Feature Augmentationcited by 116
  11. 2021Can Subnetwork Structure be the Key to Out-of-Distribution Generalization?cited by 75
  12. 2020Cars Can't Fly up in the Sky: Improving Urban-Scene Segmentation via Height-driven Attention Networkscited by 70
  13. 2021BiaSwap: Removing dataset bias with bias-tailored swapping augmentationcited by 70
  14. 2021A Winning Hand: Compressing Deep Networks Can Improve Out-Of-Distribution Robustnesscited by 62
  15. 2019Ensemble Knowledge Distillation for Learning Improved and Efficient Networkscited by 46
  16. 2021Unsupervised Learning of Debiased Representations with Pseudo-Attributescited by 44
  17. 2021Robust Contrastive Learning Using Negative Samples with Diminished Semanticscited by 43
  18. 2022Avoiding spurious correlations via logit correctioncited by 32
  19. 2023Overcoming Simplicity Bias in Deep Networks using a Feature Sievecited by 29
  20. 2023Implicit bias of SGD in L2-regularized linear DNNs: One-way jumps from high to low rankcited by 28
  21. 2022Revisiting the Importance of Amplifying Bias for Debiasingcited by 19
  22. 2022Training Debiased Subnetworks with Contrastive Weight Pruningcited by 16
  23. 2022OccamNets: Mitigating Dataset Bias by Favoring Simpler Hypothesescited by 11
  24. 2022Class Is Invariant to Context and Vice Versa: On Learning Invariance for Out-Of-Distribution Generalizationcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 4; 2026: 1.

6 papers.

  1. 2025A Closer Look at Multimodal Representation Collapsecited by 25
  2. 2025Gradient Extrapolation for Debiased Representation Learningcited by 4
  3. 2025Diffusing DeBias: Synthetic Bias Amplification for Model Debiasingcited by 3
  4. 2026BiasEdit: A Training-Free Bias-Detect-and-Edit Framework for Learning Fair Visual Classifierscited by 1
  5. 2024Towards Real-world Debiasing: Rethinking Evaluation, Challenge, and Solutioncited by 0
  6. 2025A Critical Review of Predominant Bias in Neural Networkscited by 0