mlsanityData to 7 Oct 2026

Simple data balancing achieves competitive worst-group-accuracy

arXiv 2110.14503 ↗First posted 27 Oct 2021cs.LG cs.AI cs.CRcited by 177

Not on the map. Why?

Abstract (arXiv)

We study the problem of learning classifiers that perform well across (known or unknown) groups of data. After observing that common worst-group-accuracy datasets suffer from substantial imbalances, we set out to compare state-of-the-art methods to simple balancing of classes and groups by either subsampling or reweighting data. Our results show that these data balancing baselines achieve state-of-the-art-accuracy, while being faster to train and requiring no additional hyper-parameters. In addition, we highlight that access to group information is most critical for model selection purposes, and not so much during training. All in all, our findings beg closer examination of benchmarks and methods for research in worst-group-accuracy optimization.

Key facts. Cited by 177 papers in the map (data through 7 Oct 2026). Builds on 26 of its 37 references that are in the map. First posted on arXiv 27 Oct 2021.

Builds on

26 of 37 references.

References · 26

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  4. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  5. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  6. 2019Invariant Risk Minimizationcited by 1,832
  7. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  8. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
  9. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,232
  10. 2020In Search of Lost Domain Generalizationcited by 1,042
  11. 2016Fair prediction with disparate impact: A study of bias in recidivism prediction instrumentscited by 671
  12. 2017The Implicit Bias of Gradient Descent on Separable Datacited by 649
14 more
  1. 2021Just Train Twice: Improving Group Robustness without Training Group Informationcited by 522
  2. 2019Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classificationcited by 396
  3. 2020The Pitfalls of Simplicity Bias in Neural Networkscited by 349
  4. 2020An Investigation of Why Overparameterization Exacerbates Spurious Correlationscited by 310
  5. 2020Environment Inference for Invariant Learningcited by 306
  6. 2020Gradient Starvation: A Learning Proclivity in Neural Networkscited by 262
  7. 2018What is the Effect of Importance Weighting in Deep Learning?cited by 250
  8. 2020No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problemscited by 208
  9. 2018Strike (with) a Pose: Neural Networks Are Easily Fooled by Strange Poses of Familiar Objectscited by 143
  10. 2020Learning from Failure: Training Debiased Classifier from Biased Classifiercited by 133
  11. 2014Automated Experiments on Ad Privacy Settings: A Tale of Opacity, Choice, and Discriminationcited by 83
  12. 2020Exploring Algorithmic Fairness in Robust Graph Covering Problemscited by 19
  13. 2021Predict then Interpolate: A Simple Algorithm to Learn Stable Classifierscited by 16
  14. 2021Algorithmic Bias and Data Bias: Understanding the Relation between Distributionally Robust Optimization and Data Curationcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 39; 2023: 39; 2024: 43; 2025: 35; 2026: 21.

177 papers.

Sort:

By area:

  1. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 412
  2. 2022Last Layer Re-Training is Sufficient for Robustness to Spurious Correlationscited by 379
  3. 2022Correct-N-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlationscited by 171
  4. 2022On Feature Learning in the Presence of Spurious Correlationscited by 145
  5. 2023Change is Hard: A Closer Look at Subpopulation Shiftcited by 119
  6. 2022A Whac-A-Mole Dilemma: Shortcuts Come in Multiples Where Mitigating One Amplifies Otherscited by 85
  7. 2022Domain-Adjusted Regression or: ERM May Already Learn Features Sufficient for Out-of-Distribution Generalizationcited by 69
  8. 2023Simple and Fast Group Robustness by Automatic Feature Reweightingcited by 68
  9. 2022MEDFAIR: Benchmarking Fairness for Medical Imagingcited by 66
  10. 2022Contrastive Adapters for Foundation Model Group Robustnesscited by 62
  11. 2023Towards Last-layer Retraining for Group Robustness with Fewer Annotationscited by 54
  12. 2022Improving the Fairness of Chest X-ray Classifierscited by 52
38 more
  1. 2024FairMedFM: Fairness Benchmarking for Medical Imaging Foundation Modelscited by 47
  2. 2022ID and OOD Performance Are Sometimes Inversely Correlated on Real-world Datasetscited by 44
  3. 2022ImageNet-X: Understanding Model Mistakes with Factor of Variation Annotationscited by 44
  4. 2023Spawrious: A Benchmark for Fine Control of Spurious Correlation Biasescited by 41
  5. 2024The Clever Hans Mirage: A Comprehensive Survey on Spurious Correlations in Machine Learningcited by 39
  6. 2022Characterizing Datapoints via Second-Split Forgettingcited by 37
  7. 2022Invariant Feature Learning for Generalized Long-Tailed Classificationcited by 34
  8. 2022Active Learning Helps Pretrained Models Learn the Intended Taskcited by 33
  9. 2023Feedback-guided Data Synthesis for Imbalanced Classificationcited by 33
  10. 2025Steering CLIP's vision transformer with sparse autoencoderscited by 33
  11. 2023RLSbench: Domain Adaptation Under Relaxed Label Shiftcited by 30
  12. 2023Don't blame Dataset Shift! Shortcut Learning due to Gradients and Cross Entropycited by 30
  13. 2024Calibrating Multi-modal Representations: A Pursuit of Group Robustness without Annotationscited by 30
  14. 2022Understanding Rare Spurious Correlations in Neural Networkscited by 27
  15. 2023Discovering environments with XRMcited by 26
  16. 2022Why does Throwing Away Data Improve Worst-Group Error?cited by 22
  17. 2022Mitigating Dataset Bias by Using Per-sample Gradientcited by 21
  18. 2023Rethinking the Evaluation Protocol of Domain Generalizationcited by 20
  19. 2024A Survey on Evaluation of Out-of-Distribution Generalizationcited by 20
  20. 2025Uncovering Memorization Effect in the Presence of Spurious Correlationscited by 20
  21. 2024Decomposing and Editing Predictions by Modeling Model Computationcited by 19
  22. 2022Nuisances via Negativa: Adjusting for Spurious Correlations via Data Augmentationcited by 18
  23. 2023When More is Less: Incorporating Additional Datasets Can Hurt Performance By Introducing Spurious Correlationscited by 18
  24. 2024Decompose-and-Compose: A Compositional Approach to Mitigating Spurious Correlationcited by 18
  25. 2022Distributionally Robust Models with Parametric Likelihood Ratioscited by 17
  26. 2022You Only Need a Good Embeddings Extractor to Fix Spurious Correlationscited by 17
  27. 2023Distributionally Robust Post-hoc Classifiers under Prior Shiftscited by 16
  28. 2024The Pitfalls of Memorization: When Memorization Hurts Generalizationcited by 16
  29. 2023Evaluating the Fairness of Deep Learning Uncertainty Estimates in Medical Image Analysiscited by 15
  30. 2023TTIDA: Controllable Generative Data Augmentation via Text-to-Text and Text-to-Image Modelscited by 14
  31. 2024A Textbook Remedy for Domain Shifts: Knowledge Priors for Medical Image Analysiscited by 14
  32. 2023Does Progress On Object Recognition Benchmarks Improve Real-World Generalization?cited by 13
  33. 2023Are Sex-based Physiological Differences the Cause of Gender Bias for Chest X-ray Diagnosis?cited by 13
  34. 2024Compositional Risk Minimizationcited by 13
  35. 2023SYNAuG: Exploiting Synthetic Data for Data Imbalance Problemscited by 12
  36. 2024Complexity Matters: Dynamics of Feature Learning in the Presence of Spurious Correlationscited by 12
  37. 2022Understanding new tasks through the lens of training data via exponential tiltingcited by 11
  38. 2022Simplicity Bias Leads to Amplified Performance Disparitiescited by 11

50 most cited of 177 All 177 (JSON)