mlsanityData to 14 Oct 2026

Improving Robustness to Multiple Spurious Correlations by Multi-Objective Optimization

arXiv 2409.03303 ↗First posted 5 Sep 2024cs.LG cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

We study the problem of training an unbiased and accurate model given a dataset with multiple biases. This problem is challenging since the multiple biases cause multiple undesirable shortcuts during training, and even worse, mitigating one may exacerbate the other. We propose a novel training method to tackle this challenge. Our method first groups training data so that different groups induce different shortcuts, and then optimizes a linear combination of group-wise losses while adjusting their weights dynamically to alleviate conflicts between the groups in performance; this approach, rooted in the multi-objective optimization theory, encourages to achieve the minimax Pareto solution. We also present a new benchmark with multiple biases, dubbed MultiCelebA, for evaluating debiased training methods under realistic and challenging scenarios. Our method achieved the best on three datasets with multiple biases, and also showed superior performance on conventional single-bias datasets.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 37 of its 46 references that are in the map. First posted on arXiv 5 Sep 2024.

Builds on

37 of 46 references.

References · 37

  1. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  2. 2020Shortcut Learning in Deep Neural Networkscited by 1,969
  3. 2019Invariant Risk Minimizationcited by 1,854
  4. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
  5. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 834
  6. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 821
  7. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 533
  8. 2021Just Train Twice: Improving Group Robustness without Training Group Informationcited by 523
  9. 2019Fantastic Generalization Measures and Where to Find Themcited by 513
  10. 2019Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classificationcited by 404
  11. 2022Last Layer Re-Training is Sufficient for Robustness to Spurious Correlationscited by 381
  12. 2020An Investigation of Why Overparameterization Exacerbates Spurious Correlationscited by 310
25 more
  1. 2020Environment Inference for Invariant Learningcited by 306
  2. 2022Improving Out-of-Distribution Robustness via Selective Augmentationcited by 202
  3. 2019Learning De-biased Representations with Biased Representationscited by 199
  4. 2022Correct-N-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlationscited by 172
  5. 2019REPAIR: Removing Representation Bias by Dataset Resamplingcited by 157
  6. 2020Learning from Failure: Training Debiased Classifier from Biased Classifiercited by 133
  7. 2019Learning Robust Representations by Projecting Superficial Statistics Outcited by 123
  8. 2021Explaining in Style: Training a GAN to explain a classifier in StyleSpacecited by 122
  9. 2021Learning Debiased Representation via Disentangled Feature Augmentationcited by 116
  10. 2021EnD: Entangling and Disentangling deep representations for bias correctioncited by 105
  11. 2022Spread Spurious Attribute: Improving Worst-group Accuracy with Spurious Attribute Estimationcited by 88
  12. 2022A Whac-A-Mole Dilemma: Shortcuts Come in Multiples Where Mitigating One Amplifies Otherscited by 85
  13. 2023Discover and Cure: Concept-aware Mitigation of Spurious Correlationcited by 71
  14. 2021BiaSwap: Removing dataset bias with bias-tailored swapping augmentationcited by 70
  15. 2022Contrastive Adapters for Foundation Model Group Robustnesscited by 62
  16. 2021Which Shortcut Cues Will DNNs Choose? A Study from the Parameter-Space Perspectivecited by 57
  17. 2022Learning Debiased Classifier with Biased Committeecited by 51
  18. 2022Discover and Mitigate Unknown Biases with Debiasing Alternate Networkscited by 47
  19. 2023Mitigating Spurious Correlations in Multi-modal Models during Fine-tuningcited by 47
  20. 2021Learning Fair Classifiers with Partially Annotated Group Labelscited by 40
  21. 2021PASS: Protected Attribute Suppression System for Mitigating Bias in Face Recognitioncited by 38
  22. 2021Learning Bias-Invariant Representation by Cross-Sample Mutual Information Minimizationcited by 32
  23. 2022SelecMix: Debiased Learning by Contradicting-pair Samplingcited by 29
  24. 2020Latent Adversarial Debiasing: Mitigating Collider Bias in Deep Neural Networkscited by 16
  25. 2021Robust Optimization for Multilingual Translation with Imbalanced Datacited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 2.

6 papers.

  1. 2025Mitigating Spurious Correlation via Distributionally Robust Learning with Hierarchical Ambiguity Setscited by 8
  2. 2025Subgroups Matter for Robust Bias Mitigationcited by 2
  3. 2025Single-weight Model Editing for Post-hoc Spurious Correlation Neutralizationcited by 1
  4. 2025GroupCoOp: Group-robust Fine-tuning via Group Prompt Learningcited by 0
  5. 2026Bias In, Bias Out? Finding Unbiased Subnetworks in Vanilla Modelscited by 0
  6. 2026Optimal Transport Reweighting for Robust Learning under Spurious Correlations and Label Noisecited by 0