mlsanityData to 7 Oct 2026

Learning the Robustness Mechanism with Bilevel Optimization

arXiv 2609.35541 ↗First posted 28 Sep 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We propose a distributionally robust learning framework where parameters defining the robustness mechanism are learned from held-out data instead of extensively tuned. Using bilevel optimization with both upper and lower level minimax problems, we create two instances of our framework to tackle setups with and without group labels in the training set. Theoretically, we provide sample complexity analysis for our robustness mechanism learning paradigm, showing that it achieves generalization guarantees comparable to exhaustive grid search while being more computationally efficient. Empirically, we evaluate our framework under a challenging setup when both intra-group and inter-group test distribution shifts occur at the same time, thereby demonstrating the efficacy and scalability of our method.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 57 of its 73 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

57 of 73 references.

References · 57

  1. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  2. 2019Invariant Risk Minimizationcited by 1,832
  3. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  4. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
  5. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,232
  6. 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 806
  7. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 737
  8. 2018Learning to Reweight Examples for Robust Deep Learningcited by 670
  9. 2019Meta-Learning with Implicit Gradientscited by 523
  10. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 431
  11. 2018Bilevel Programming for Hyperparameter Optimization and Meta-Learningcited by 403
  12. 2018Meta-learning with differentiable closed-form solverscited by 395
38 more
  1. 2022Last Layer Re-Training is Sufficient for Robustness to Spurious Correlationscited by 379
  2. 2018Confounding variables can degrade generalization performance of radiological deep learning modelscited by 374
  3. 2015Gradient-based Hyperparameter Optimization through Reversible Learningcited by 357
  4. 2018Learning Models with Uniform Performance via Distributionally Robust Optimizationcited by 334
  5. 2020Environment Inference for Invariant Learningcited by 306
  6. 2019Optimizing Millions of Hyperparameters by Implicit Differentiationcited by 303
  7. 2017Forward and Reverse Gradient-Based Hyperparameter Optimizationcited by 232
  8. 2020No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problemscited by 208
  9. 2018Truncated Back-propagation for Bilevel Optimizationcited by 187
  10. 2016Statistics of Robust Optimization: A Generalized Empirical Likelihood Approachcited by 182
  11. 2022Correct-N-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlationscited by 171
  12. 2022On Feature Learning in the Presence of Spurious Correlationscited by 145
  13. 2020A Two-Timescale Framework for Bilevel Optimization: Complexity Analysis and Application to Actor-Criticcited by 141
  14. 2023Change is Hard: A Closer Look at Subpopulation Shiftcited by 119
  15. 2017Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenescited by 105
  16. 2022Spread Spurious Attribute: Improving Worst-group Accuracy with Spurious Attribute Estimationcited by 88
  17. 2025Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyondcited by 76
  18. 2023Discover and Cure: Concept-aware Mitigation of Spurious Correlationcited by 71
  19. 2023Simple and Fast Group Robustness by Automatic Feature Reweightingcited by 68
  20. 2022MaskTune: Mitigating Spurious Correlations by Forcing to Explorecited by 66
  21. 2023Towards Last-layer Retraining for Group Robustness with Fewer Annotationscited by 54
  22. 2024Towards Robust Alignment of Language Models: Distributionally Robustifying Direct Preference Optimizationcited by 52
  23. 2018Deep Learning Predicts Hip Fracture using Confounding Patient and Healthcare Variablescited by 50
  24. 2021A Theory of Label Propagation for Subpopulation Shiftcited by 48
  25. 2021Unsupervised Learning of Debiased Representations with Pseudo-Attributescited by 44
  26. 2023First-order penalty methods for bilevel optimizationcited by 40
  27. 2023Robust Learning with Progressive Data Expansion Against Spurious Correlationcited by 39
  28. 2024Getting More Juice Out of the SFT Data: Reward Learning from Human Demonstration Improves SFT for LLM Alignmentcited by 32
  29. 2023Discovering environments with XRMcited by 26
  30. 2021Focus on the Common Good: Group Distributional Robustness Followscited by 25
  31. 2022SAPD+: An Accelerated Stochastic Method for Nonconvex-Concave Minimax Problemscited by 23
  32. 2024A Survey on Evaluation of Out-of-Distribution Generalizationcited by 20
  33. 2023Evaluating the Fairness of Deep Learning Uncertainty Estimates in Medical Image Analysiscited by 15
  34. 2024Improving Group Robustness on Spurious Correlation Requires Preciser Group Inferencecited by 15
  35. 2023Distributionally Robust Optimization with Probabilistic Groupcited by 14
  36. 2022AGRO: Adversarial Discovery of Error-prone groups for Robust Optimizationcited by 10
  37. 2024Bilevel reinforcement learning via the development of hyper-gradient without lower-level convexitycited by 10
  38. 2025Mitigating Spurious Correlation via Distributionally Robust Learning with Hierarchical Ambiguity Setscited by 7

50 most cited of 57 All 57 (JSON)

Built on by

Not cited yet.