mlsanityData to 7 Oct 2026

Min-Max Bilevel Multi-objective Optimization with Applications in Machine Learning

arXiv 2203.01924 ↗First posted 3 Mar 2022cs.LG math.OCcited by 9

Not on the map. Why?

Abstract (arXiv)

We consider a generic min-max multi-objective bilevel optimization problem with applications in robust machine learning such as representation learning and hyperparameter optimization. We design MORBiT, a novel single-loop gradient descent-ascent bilevel optimization algorithm, to solve the generic problem and present a novel analysis showing that MORBiT converges to the first-order stationary point at a rate of widetildeO(n1/2 K-2/5) for a class of weakly convex problems with n objectives upon K iterations of the algorithm. Our analysis utilizes novel results to handle the non-smooth min-max multi-objective setup and to obtain a sublinear dependence in the number of objectives n. Experimental results on robust representation learning and robust hyperparameter optimization showcase (i) the advantages of considering the min-max multi-objective setup, and (ii) convergence properties of the proposed MORBiT. Our code is at https://github.com/minimario/MORBiT.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 22 of its 61 references that are in the map. First posted on arXiv 3 Mar 2022.

Builds on

22 of 61 references.

References · 22

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  4. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  5. 2019Meta-Learning with Implicit Gradientscited by 523
  6. 2018Bilevel Programming for Hyperparameter Optimization and Meta-Learningcited by 403
  7. 2019Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAMLcited by 359
  8. 2018Learning Models with Uniform Performance via Distributionally Robust Optimizationcited by 334
  9. 2019On Gradient Descent Ascent for Nonconvex-Concave Minimax Problemscited by 266
  10. 2018Approximation Methods for Bilevel Programmingcited by 231
  11. 2020Bilevel Optimization: Convergence Analysis and Enhanced Designcited by 219
  12. 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 192
10 more
  1. 2019A Generalized Algorithm for Multi-Objective Reinforcement Learning and Policy Adaptationcited by 166
  2. 2021Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyondcited by 151
  3. 2016On Differentiating Parameterized Argmin and Argmax Problems with Application to Bi-level Optimizationcited by 108
  4. 2021A Single-Timescale Method for Stochastic Bilevel Optimizationcited by 69
  5. 2019Hybrid Block Successive Approximation for One-Sided Non-Convex Min-Max Problems: Algorithms and Applicationscited by 55
  6. 2020Provable Representation Learning for Imitation Learning via Bi-level Optimizationcited by 41
  7. 2021LassoBench: A High-Dimensional Hyperparameter Optimization Benchmark Suite for Lassocited by 41
  8. 2018Stochastic subgradient method converges at the rate O(k-1/4) on weakly convex functionscited by 31
  9. 2016Minimizing the Maximal Loss: How and Why?cited by 30
  10. 2020Learning to Generate Image Source-Agnostic Universal Adversarial Perturbationscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 2; 2024: 2; 2025: 1; 2026: 2.

9 papers.

  1. 2023Optimal Algorithms for Stochastic Bilevel Optimization under Relaxed Smoothness Conditionscited by 31
  2. 2023Refined Coreset Selection: Towards Minimal Coreset Size under Model Performance Constraintscited by 13
  3. 2022Multi-block Min-max Bilevel Optimization with Applications in Multi-task Deep AUC Maximizationcited by 11
  4. 2022RoCourseNet: Distributionally Robust Training of a Prediction Aware Recourse Modelcited by 11
  5. 2024A First-Order Multi-Gradient Algorithm for Multi-Objective Bi-Level Optimizationcited by 8
  6. 2024Partial Rankings of Optimizerscited by 8
  7. 2025Multi-Objective Bilevel Learningcited by 2
  8. 2026A Tale of Two Problems: Multi-Task Bilevel Learning Meets Equality Constrained Multi-Objective Optimizationcited by 1
  9. 2026On the Stability and Generalization of First-order Bilevel Minimax Optimizationcited by 0