mlsanityData to 7 Oct 2026

On Differentiating Parameterized Argmin and Argmax Problems with Application to Bi-level Optimization

arXiv 1607.05447 ↗First posted 19 Jul 2016cs.CV math.OCcited by 108

Not on the map. Why?

Abstract (arXiv)

Some recent works in machine learning and computer vision involve the solution of a bi-level optimization problem. Here the solution of a parameterized lower-level problem binds variables that appear in the objective of an upper-level problem. The lower-level problem typically appears as an argmin or argmax optimization problem. Many techniques have been proposed to solve bi-level optimization problems, including gradient descent, which is popular with current end-to-end learning approaches. In this technical report we collect some results on differentiating argmin and argmax optimization problems with and without constraints and provide some insightful motivating examples.

Key facts. Cited by 108 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 Jul 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 24; 2022: 21; 2023: 22; 2024: 21; 2025: 6; 2026: 14.

108 papers.

Sort:

By area:

  1. 2021Graph Contrastive Learning Automatedcited by 287
  2. 2023Model Sparsity Can Simplify Machine Unlearningcited by 206
  3. 2021Efficient and Modular Implicit Differentiationcited by 199
  4. 2021Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyondcited by 151
  5. 2021Hyperbolic Deep Neural Networks: A Surveycited by 138
  6. 2021Sinkformers: Transformers with Doubly Stochastic Attentioncited by 136
  7. 2023Decision-Focused Learning: Foundations, State of the Art, Benchmark and Future Opportunitiescited by 129
  8. 2021Counterfactual Explanations Can Be Manipulatedcited by 113
  9. 2021Provably Faster Algorithms for Bilevel Optimizationcited by 106
  10. 2022Equivariance with Learned Canonicalization Functionscited by 106
  11. 2024SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearningcited by 104
  12. 2022Tutorial on amortized optimizationcited by 83
38 more
  1. 2021Revisiting and Advancing Fast Adversarial Training Through The Lens of Bi-Level Optimizationcited by 75
  2. 2022Theseus: A Library for Differentiable Nonlinear Optimizationcited by 64
  3. 2021Amortized Implicit Differentiation for Stochastic Bilevel Optimizationcited by 62
  4. 2022Advancing Model Pruning via Bi-level Optimizationcited by 55
  5. 2021CombOptNet: Fit the Right NP-Hard Problem by Learning Integer Programming Constraintscited by 49
  6. 2023On Finding Small Hyper-Gradients in Bilevel Optimization: Hardness Results and Improved Analysiscited by 39
  7. 2023An Introduction to Bi-level Optimization: Foundations and Applications in Signal Processing and Machine Learningcited by 39
  8. 2022DDG-DA: Data Distribution Generation for Predictable Concept Drift Adaptationcited by 36
  9. 2021On the Convergence Theory for Hessian-Free Bilevel Algorithmscited by 33
  10. 2022PyEPO: A PyTorch-based End-to-End Predict-then-Optimize Library for Linear and Integer Programmingcited by 32
  11. 2023Optimal Algorithms for Stochastic Bilevel Optimization under Relaxed Smoothness Conditionscited by 31
  12. 2021Max-Margin Contrastive Learningcited by 27
  13. 2024WAGLE: Strategic Weight Attribution for Effective and Modular Unlearning in Large Language Modelscited by 25
  14. 2021Implicit differentiation for fast hyperparameter selection in non-smooth convex learningcited by 22
  15. 2022Alternating Differentiation for Optimization Layerscited by 22
  16. 2022Decentralized Stochastic Bilevel Optimization with Improved per-Iteration Complexitycited by 21
  17. 2024Functional Bilevel Optimization for Machine Learningcited by 20
  18. 2022Efficiently Escaping Saddle Points in Bilevel Optimizationcited by 19
  19. 2021Multiset-Equivariant Set Prediction with Approximate Implicit Differentiationcited by 17
  20. 2022A Conditional Gradient-based Method for Simple Bilevel Optimization with Convex Lower-level Problemcited by 17
  21. 2023Landscape Surrogate: Learning Decision Losses for Mathematical Optimization Under Partial Informationcited by 17
  22. 2023Unlocking Slot Attention by Changing Optimal Transport Costscited by 16
  23. 2023Behavior Alignment via Reward Function Optimizationcited by 16
  24. 2024Adversarial Machine Unlearningcited by 16
  25. 2024Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomycited by 16
  26. 2023Efficient Gradient Approximation Method for Constrained Bilevel Optimizationcited by 15
  27. 2021Sign-MAML: Efficient Model-Agnostic Meta-Learning by SignSGDcited by 14
  28. 2023SimFBO: Towards Simple, Flexible and Communication-efficient Federated Bilevel Learningcited by 14
  29. 2024Guiding Neural Collapse: Optimising Towards the Nearest Simplex Equiangular Tight Framecited by 14
  30. 2022Asynchronous Distributed Bilevel Optimizationcited by 13
  31. 2023Backpropagation of Unrolled Solvers with Folded Optimizationcited by 13
  32. 2023Communication-Efficient Federated Hypergradient Computation via Aggregated Iterative Differentiationcited by 13
  33. 2022A Primal-Dual Approach to Bilevel Optimization with Multiple Inner Minimacited by 12
  34. 2021Convergent Graph Solverscited by 11
  35. 2021Neural Fixed-Point Acceleration for Convex Optimizationcited by 10
  36. 2022Alternating Implicit Projected SGD and Its Efficient Variants for Equality-constrained Bilevel Optimizationcited by 10
  37. 2023What Is Missing in IRM Training and Evaluation? Challenges and Solutionscited by 10
  38. 2024First-Order Methods for Linearly Constrained Bilevel Optimizationcited by 10

50 most cited of 108 All 108 (JSON)