mlsanityData to 7 Oct 2026

Combining Diverse Feature Priors

arXiv 2110.08220 ↗First posted 15 Oct 2021cs.LG cs.CVcited by 12

Not on the map. Why?

Abstract (arXiv)

To improve model generalization, model designers often restrict the features that their models use, either implicitly or explicitly. In this work, we explore the design space of leveraging such feature priors by viewing them as distinct perspectives on the data. Specifically, we find that models trained with diverse sets of feature priors have less overlapping failure modes, and can thus be combined more effectively. Moreover, we demonstrate that jointly training such models on additional (unlabeled) data allows them to correct each other's mistakes, which, in turn, leads to better generalization and resilience to spurious correlations. Code available at https://github.com/MadryLab/copriors

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 56 of its 90 references that are in the map. First posted on arXiv 15 Oct 2021.

Builds on

56 of 90 references.

References · 56

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  5. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  6. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  7. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  8. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,080
  9. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  10. 2019Invariant Risk Minimizationcited by 1,832
  11. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  12. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
38 more
  1. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,390
  2. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  3. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
  4. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,232
  5. 2017Deep Image Priorcited by 1,097
  6. 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,054
  7. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,053
  8. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,010
  9. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
  10. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 918
  11. 2018Robustness May Be at Odds with Accuracycited by 873
  12. 2018Annotation Artifacts in Natural Language Inference Datacited by 656
  13. 2018Recognition in Terra Incognitacited by 655
  14. 2019Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learningcited by 507
  15. 2018Certified Robustness to Adversarial Examples with Differential Privacycited by 462
  16. 2019Domain Generalization by Solving Jigsaw Puzzlescited by 449
  17. 2013DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognitioncited by 422
  18. 2019Confidence Regularized Self-Trainingcited by 418
  19. 2020Noise or Signal: The Role of Image Backgrounds in Object Recognitioncited by 342
  20. 2020Do Adversarially Robust ImageNet Models Transfer Better?cited by 315
  21. 2021In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learningcited by 301
  22. 2020Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learningcited by 292
  23. 2019A Fourier Perspective on Model Robustness in Computer Visioncited by 288
  24. 2019Label Propagation for Deep Semi-supervised Learningcited by 287
  25. 2019Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNetcited by 260
  26. 2020Theoretical Analysis of Self-Training with Deep Networks on Unlabeled Datacited by 179
  27. 2020Self-Distillation Amplifies Regularization in Hilbert Spacecited by 171
  28. 2020Understanding Self-Training for Gradual Domain Adaptationcited by 149
  29. 2020Hyperparameter Ensembles for Robustness and Uncertainty Quantificationcited by 149
  30. 2018Deep Co-Training for Semi-Supervised Image Recognitioncited by 145
  31. 2019Adversarial Examples Are a Natural Consequence of Test Error in Noisecited by 114
  32. 2020SENTRY: Selective Entropy Optimization via Committee Consistency for Unsupervised Domain Adaptationcited by 78
  33. 2020Shape-Texture Debiased Neural Network Trainingcited by 76
  34. 2021On the effectiveness of adversarial training against common corruptionscited by 71
  35. 2020Self-training Avoids Using Spurious Features Under Domain Shiftcited by 69
  36. 2020Informative Dropout for Robust Representation Learning: A Shape-bias Perspectivecited by 57
  37. 2020Frustratingly Simple Domain Generalization via Image Stylizationcited by 43
  38. 2019Are Perceptually-Aligned Gradients a General Property of Robust Classifiers?cited by 42

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 7; 2024: 2; 2025: 1.

12 papers.

  1. 2022Model Ratatouille: Recycling Diverse Models for Out-of-Distribution Generalizationcited by 97
  2. 2023Spurious Feature Diversification Improves Out-of-distribution Generalizationcited by 26
  3. 2025ImageNet-trained CNNs are not biased towards texture: Revisiting feature reliance through controlled suppressioncited by 13
  4. 2024Meta-Exploiting Frequency Prior for Cross-Domain Few-Shot Learningcited by 11
  5. 2023Rapid Network Adaptation: Learning to Adapt Neural Networks Using Test-Time Feedbackcited by 10
  6. 2022InBiaseD: Inductive Bias Distillation to Improve Generalization and Robustness through Shape-awarenesscited by 5
  7. 2023Towards Understanding How Self-training Tolerates Data Backdoor Poisoningcited by 4
  8. 2023LSFSL: Leveraging Shape Information in Few-shot Learningcited by 4
  9. 2023Towards Brain Inspired Design for Addressing the Shortcomings of ANNscited by 1
  10. 2024Can We Break Free from Strong Data Augmentations in Self-Supervised Learning?cited by 1
  11. 2023Diverse Audio Embeddings -- Bringing Features Back Outperforms CLAP!cited by 0
  12. 2023Something for (almost) nothing: Improving deep ensemble calibration using unlabeled datacited by 0