# Differentiating Nonsmooth Solutions to Parametric Monotone Inclusion Problems

arXiv: https://arxiv.org/abs/2212.07844 · HTML: https://mlsanity.com/paper/2212.07844/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 80 references that are in the map.
- First posted on arXiv 15 Dec 2022.

## Builds on (most cited first, 30 of 31)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [OptNet: Differentiable Optimization as a Layer in Neural Networks](https://mlsanity.com/paper/1703.00443/) (2017)
- [Optimizing Millions of Hyperparameters by Implicit Differentiation](https://mlsanity.com/paper/1911.02590/) (2019)
- [Hyperparameter optimization with approximate gradient](https://mlsanity.com/paper/1602.02355/) (2016)
- [Efficient and Modular Implicit Differentiation](https://mlsanity.com/paper/2105.15183/) (2021)
- [Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyond](https://mlsanity.com/paper/2101.11517/) (2021)
- [Monotone operator equilibrium networks](https://mlsanity.com/paper/2006.08591/) (2020)
- [Conservative set valued fields, automatic differentiation, stochastic gradient method and deep learning](https://mlsanity.com/paper/1909.10300/) (2019)
- [Stochastic Variance Reduction Methods for Saddle-Point Problems](https://mlsanity.com/paper/1605.06398/) (2016)
- [A mathematical model for automatic differentiation in machine learning](https://mlsanity.com/paper/2006.02080/) (2020)
- [From error bounds to the complexity of first-order descent methods for convex functions](https://mlsanity.com/paper/1510.08234/) (2015)
- [Nonsmooth Implicit Differentiation for Machine Learning and Optimization](https://mlsanity.com/paper/2106.04350/) (2021)
- [A Forward-Backward Splitting Method for Monotone Inclusions Without Cocoercivity](https://mlsanity.com/paper/1808.04162/) (2018)
- [Implicit differentiation of Lasso-type models for hyperparameter optimization](https://mlsanity.com/paper/2002.08943/) (2020)
- [Automatic differentiation of nonsmooth iterative algorithms](https://mlsanity.com/paper/2206.00457/) (2022)
- [Automatic Differentiation of Some First-Order Methods in Parametric Optimization](https://mlsanity.com/paper/1910.05696/) (2019)
- [A Primal-Dual Algorithm with Line Search for General Convex-Concave Saddle Point Problems](https://mlsanity.com/paper/1803.01401/) (2018)
- [Convergence of the Forward-Backward Algorithm: Beyond the Worst Case with the Help of Geometry](https://mlsanity.com/paper/1703.09477/) (2017)
- [Accelerated Bregman Primal-Dual methods applied to Optimal Transport and Wasserstein Barycenter problems](https://mlsanity.com/paper/2203.00802/) (2022)
- [ℓ0 Sparsifying Transform Learning with Efficient Optimal Updates and Convergence Guarantees](https://mlsanity.com/paper/1501.02859/) (2015)
- [Local Linear Convergence Analysis of Primal-Dual Splitting Methods](https://mlsanity.com/paper/1705.01926/) (2017)
- [Sensitivity Analysis for Mirror-Stratifiable Convex Functions](https://mlsanity.com/paper/1707.03194/) (2017)
- [A Stochastic Bregman Primal-Dual Splitting Algorithm for Composite Optimization](https://mlsanity.com/paper/2112.11928/) (2021)
- [Fixed-Point Automatic Differentiation of Forward--Backward Splitting Algorithms for Partly Smooth Functions](https://mlsanity.com/paper/2208.03107/) (2022)
- [An inertial forward-backward-forward primal-dual splitting algorithm for solving monotone inclusion problems](https://mlsanity.com/paper/1402.5291/) (2014)
- [Inertial Douglas-Rachford splitting for monotone inclusion problems](https://mlsanity.com/paper/1403.3330/) (2014)
- [Perturbation techniques for convergence analysis of proximal gradient method and other first-order algorithms via variational analysis](https://mlsanity.com/paper/1810.10051/) (2018)
- [From resolvents to generalized equations and quasi-variational inequalities: existence and differentiability](https://mlsanity.com/paper/2109.13569/) (2021)
- [Supervised learning of analysis-sparsity priors with automatic differentiation](https://mlsanity.com/paper/2112.07990/) (2021)

## Built on by (most cited first, 5 of 5)

- [Adam-family Methods with Decoupled Weight Decay in Deep Learning](https://mlsanity.com/paper/2310.08858/) (2023)
- [Stability of Data-Dependent Ridge-Regularization for Inverse Problems](https://mlsanity.com/paper/2406.12289/) (2024)
- [Nonsmooth Implicit Differentiation: Deterministic and Stochastic Convergence Rates](https://mlsanity.com/paper/2403.11687/) (2024)
- [Learning truly monotone operators with applications to nonlinear inverse problems](https://mlsanity.com/paper/2404.00390/) (2024)
- [Developing Lagrangian-based Methods for Nonsmooth Nonconvex Optimization](https://mlsanity.com/paper/2404.09438/) (2024)

## Abstract (arXiv)

We leverage path differentiability and a recent result on nonsmooth implicit differentiation calculus to give sufficient conditions ensuring that the solution to a monotone inclusion problem will be path differentiable, with formulas for computing its generalized gradient. A direct consequence of our result is that these solutions happen to be differentiable almost everywhere. Our approach is fully compatible with automatic differentiation and comes with assumptions which are easy to check, roughly speaking: semialgebraicity and strong monotonicity. We illustrate the scope of our results by considering three fundamental composite problem settings: strongly convex problems, dual solutions to convex minimization problems and primal-dual solutions to min-max problems.