mlsanityData to 7 Oct 2026

Coping with Label Shift via Distributionally Robust Optimisation

arXiv 2010.12230 ↗First posted 23 Oct 2020cs.LG cs.CV math.OCcited by 69

Not on the map. Why?

Abstract (arXiv)

The label shift problem refers to the supervised learning setting where the train and test label distributions do not match. Existing work addressing label shift usually assumes access to an unlabelled test sample. This sample may be used to estimate the test label distribution, and to then train a suitably re-weighted classifier. While approaches using this idea have proven effective, their scope is limited as it is not always feasible to access the target domain; further, they require repeated retraining if the model is to be deployed in multiple test environments. Can one instead learn a single classifier that is robust to arbitrary label shifts from a broad family? In this paper, we answer this question by proposing a model that minimises an objective based on distributionally robust optimisation (DRO). We then design and analyse a gradient descent-proximal mirror ascent algorithm tailored for large-scale problems to optimise the proposed objective. %, and establish its convergence. Finally, through experiments on CIFAR-100 and ImageNet, we show that our technique can significantly improve performance over a number of baselines in settings where label shift is present.

Key facts. Cited by 69 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 19; 2023: 22; 2024: 8; 2025: 13; 2026: 2.

69 papers.

Sort:

By area:

  1. 2021Just Train Twice: Improving Group Robustness without Training Group Informationcited by 522
  2. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 412
  3. 2022Last Layer Re-Training is Sufficient for Robustness to Spurious Correlationscited by 379
  4. 2022Improving Out-of-Distribution Robustness via Selective Augmentationcited by 202
  5. 2022Wild-Time: A Benchmark of in-the-Wild Distribution Shift over Timecited by 85
  6. 2023Discover and Cure: Concept-aware Mitigation of Spurious Correlationcited by 71
  7. 2023Benchmarking Distribution Shift in Tabular Data with TableShiftcited by 64
  8. 2022DrugOOD: Out-of-Distribution (OOD) Dataset Curator and Benchmark for AI-aided Drug Discovery -- A Focus on Affinity Prediction Problems with Noise Annotationscited by 54
  9. 2023Robust and Interpretable Medical Image Classifiers via Concept Bottleneck Modelscited by 40
  10. 2021Non-convex Distributionally Robust Optimization: Non-asymptotic Analysiscited by 39
  11. 2023Robust Learning with Progressive Data Expansion Against Spurious Correlationcited by 39
  12. 2025Representation Learning for Tabular Data: A Comprehensive Surveycited by 36
38 more
  1. 2025Steering Out-of-Distribution Generalization with Concept Ablation Fine-Tuningcited by 34
  2. 2022Domain Adaptation under Open Set Label Shiftcited by 32
  3. 2023RLSbench: Domain Adaptation Under Relaxed Label Shiftcited by 30
  4. 2021BARACK: Partially Supervised Group Robustness With Guaranteescited by 25
  5. 2022Explicit Tradeoffs between Adversarial and Natural Distributional Robustnesscited by 20
  6. 2022Distributionally Robust Bayesian Optimization with varphi-divergencescited by 16
  7. 2022Near-Optimal Algorithms for Group Distributionally Robust Optimization and Beyondcited by 16
  8. 2023Distributionally Robust Post-hoc Classifiers under Prior Shiftscited by 16
  9. 2023Distributionally Robust Optimization with Probabilistic Groupcited by 14
  10. 2022Spuriosity Rankings: Sorting Data to Measure and Mitigate Biasescited by 13
  11. 2023Optimal Multi-Distribution Learningcited by 13
  12. 2022Simplicity Bias Leads to Amplified Performance Disparitiescited by 11
  13. 2024Do causal predictors generalize better to new domains?cited by 11
  14. 2022What You See is What You Get: Principled Deep Learning via Distributional Generalizationcited by 10
  15. 2025Spurious Correlations in High Dimensional Regression: The Roles of Regularization, Simplicity Bias and Over-Parameterizationcited by 9
  16. 2024Constructing Concept-based Models to Mitigate Spurious Correlations with Minimal Human Effortcited by 7
  17. 2021Robust Generalization despite Distribution Shift via Minimum Discriminating Informationcited by 6
  18. 2022Towards Group Robustness in the presence of Partial Group Labelscited by 6
  19. 2022Expert-Calibrated Learning for Online Optimization with Switching Costscited by 6
  20. 2023Bias Amplification Enhances Minority Group Performancecited by 6
  21. 2023Invariant Feature Regularization for Fair Face Recognitioncited by 6
  22. 2021Adversarial Regression with Doubly Non-negative Weighting Matricescited by 5
  23. 2022Learning Distributionally Robust Models at Scale via Composite Optimizationcited by 5
  24. 2022Learning with Noisy Labels over Imbalanced Subpopulationscited by 5
  25. 2023ASPIRE: Language-Guided Data Augmentation for Improving Robustness Against Spurious Correlationscited by 5
  26. 2023Complementary Benefits of Contrastive Learning and Self-Training Under Distribution Shiftcited by 5
  27. 2025When Shift Happens - Confounding Is to Blamecited by 5
  28. 2022Improving group robustness under noisy labels using predictive uncertaintycited by 4
  29. 2023Stochastic Approximation Approaches to Group Distributionally Robust Optimization and Beyondcited by 4
  30. 2023Robustified Learning for Online Optimization with Memory Costscited by 4
  31. 2023Distributionally Robust Learning for Multi-source Unsupervised Domain Adaptationcited by 4
  32. 2023Unraveling the Key Components of OOD Generalization via Diversificationcited by 4
  33. 2023DRAUC: An Instance-wise Distributionally Robust AUC Optimization Frameworkcited by 3
  34. 2024Rejection via Learning Density Ratioscited by 3
  35. 2025Group-robust Machine Unlearningcited by 3
  36. 2022Tailoring to the Tails: Risk Measures for Fine-Grained Tail Sensitivitycited by 2
  37. 2023Pushing the Accuracy-Group Robustness Frontier with Introspective Self-playcited by 2
  38. 2024Towards Utilising a Range of Neural Activations for Comprehending Representational Associationscited by 2

50 most cited of 69 All 69 (JSON)