mlsanityData to 14 Oct 2026

Differentiating Nonsmooth Solutions to Parametric Monotone Inclusion Problems

arXiv 2212.07844 ↗First posted 15 Dec 2022cs.LG math.OCcited by 5

Not on the map. Why?

Abstract (arXiv)

We leverage path differentiability and a recent result on nonsmooth implicit differentiation calculus to give sufficient conditions ensuring that the solution to a monotone inclusion problem will be path differentiable, with formulas for computing its generalized gradient. A direct consequence of our result is that these solutions happen to be differentiable almost everywhere. Our approach is fully compatible with automatic differentiation and comes with assumptions which are easy to check, roughly speaking: semialgebraicity and strong monotonicity. We illustrate the scope of our results by considering three fundamental composite problem settings: strongly convex problems, dual solutions to convex minimization problems and primal-dual solutions to min-max problems.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 31 of its 80 references that are in the map. First posted on arXiv 15 Dec 2022.

Builds on

31 of 80 references.

References · 31

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  3. 2017OptNet: Differentiable Optimization as a Layer in Neural Networkscited by 599
  4. 2019Optimizing Millions of Hyperparameters by Implicit Differentiationcited by 304
  5. 2016Hyperparameter optimization with approximate gradientcited by 233
  6. 2021Efficient and Modular Implicit Differentiationcited by 203
  7. 2021Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyondcited by 153
  8. 2020Monotone operator equilibrium networkscited by 100
  9. 2019Conservative set valued fields, automatic differentiation, stochastic gradient method and deep learningcited by 72
  10. 2016Stochastic Variance Reduction Methods for Saddle-Point Problemscited by 60
  11. 2020A mathematical model for automatic differentiation in machine learningcited by 45
  12. 2015From error bounds to the complexity of first-order descent methods for convex functionscited by 43
19 more
  1. 2021Nonsmooth Implicit Differentiation for Machine Learning and Optimizationcited by 43
  2. 2018A Forward-Backward Splitting Method for Monotone Inclusions Without Cocoercivitycited by 36
  3. 2020Implicit differentiation of Lasso-type models for hyperparameter optimizationcited by 34
  4. 2022Automatic differentiation of nonsmooth iterative algorithmscited by 25
  5. 2019Automatic Differentiation of Some First-Order Methods in Parametric Optimizationcited by 18
  6. 2018A Primal-Dual Algorithm with Line Search for General Convex-Concave Saddle Point Problemscited by 9
  7. 2017Convergence of the Forward-Backward Algorithm: Beyond the Worst Case with the Help of Geometrycited by 7
  8. 2022Accelerated Bregman Primal-Dual methods applied to Optimal Transport and Wasserstein Barycenter problemscited by 7
  9. 2015ℓ0 Sparsifying Transform Learning with Efficient Optimal Updates and Convergence Guaranteescited by 6
  10. 2017Local Linear Convergence Analysis of Primal-Dual Splitting Methodscited by 4
  11. 2017Sensitivity Analysis for Mirror-Stratifiable Convex Functionscited by 4
  12. 2021A Stochastic Bregman Primal-Dual Splitting Algorithm for Composite Optimizationcited by 4
  13. 2022Fixed-Point Automatic Differentiation of Forward--Backward Splitting Algorithms for Partly Smooth Functionscited by 4
  14. 2014An inertial forward-backward-forward primal-dual splitting algorithm for solving monotone inclusion problemscited by 3
  15. 2014Inertial Douglas-Rachford splitting for monotone inclusion problemscited by 3
  16. 2018Perturbation techniques for convergence analysis of proximal gradient method and other first-order algorithms via variational analysiscited by 1
  17. 2021From resolvents to generalized equations and quasi-variational inequalities: existence and differentiabilitycited by 1
  18. 2021Supervised learning of analysis-sparsity priors with automatic differentiationcited by 1
  19. 2022LASSO reloaded: a variational analysis perspective with applications to compressed sensingcited by 1

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 4.

5 papers.

  1. 2023Adam-family Methods with Decoupled Weight Decay in Deep Learningcited by 5
  2. 2024Stability of Data-Dependent Ridge-Regularization for Inverse Problemscited by 5
  3. 2024Nonsmooth Implicit Differentiation: Deterministic and Stochastic Convergence Ratescited by 2
  4. 2024Learning truly monotone operators with applications to nonlinear inverse problemscited by 2
  5. 2024Developing Lagrangian-based Methods for Nonsmooth Nonconvex Optimizationcited by 2