# Recursive Bound-Constrained AdaGrad with Applications to Multilevel and Domain Decomposition Minimization

arXiv: https://arxiv.org/abs/2507.11513 · HTML: https://mlsanity.com/paper/2507.11513/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 80 references that are in the map.
- First posted on arXiv 15 Jul 2025.

## Builds on (most cited first, 28 of 28)

- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Optimization Methods for Large-Scale Machine Learning](https://mlsanity.com/paper/1606.04838/) (2016)
- [Stable Architectures for Deep Neural Networks](https://mlsanity.com/paper/1705.03341/) (2017)
- [Physics-informed neural networks with hard constraints for inverse design](https://mlsanity.com/paper/2102.04626/) (2021)
- [Multi-level Residual Networks from Dynamical Systems View](https://mlsanity.com/paper/1710.10348/) (2017)
- [Continuous-in-Depth Neural Networks](https://mlsanity.com/paper/2008.02389/) (2020)
- [Global convergence rate analysis of unconstrained optimization methods based on probabilistic models](https://mlsanity.com/paper/1505.06070/) (2015)
- [Enhancing training of physics-informed neural networks using domain-decomposition based preconditioning strategies](https://mlsanity.com/paper/2306.17648/) (2023)
- [Global Convergence Rate Analysis of a Generic Line Search Algorithm with Noise](https://mlsanity.com/paper/1910.04055/) (2019)
- [Multilevel Minimization for Deep Residual Networks](https://mlsanity.com/paper/2004.06196/) (2020)
- [Globally Convergent Multilevel Training of Deep Residual Networks](https://mlsanity.com/paper/2107.07572/) (2021)
- [A Block-Coordinate Approach of Multi-level Optimization with an Application to Physics-Informed Neural Networks](https://mlsanity.com/paper/2305.14477/) (2023)
- [Multilevel Objective-Function-Free Optimization with an Application to Neural Networks Training](https://mlsanity.com/paper/2302.07049/) (2023)
- [Two-level overlapping additive Schwarz preconditioner for training scientific machine learning applications](https://mlsanity.com/paper/2406.10997/) (2024)
- [Additive Schwarz Methods for Convex Optimization as Gradient Methods](https://mlsanity.com/paper/1912.03617/) (2019)
- [The Impact of Noise on Evaluation Complexity: The Deterministic Trust-Region Case](https://mlsanity.com/paper/2104.02519/) (2021)
- [Better Training using Weight-Constrained Stochastic Dynamics](https://mlsanity.com/paper/2106.10704/) (2021)
- [On the Globalization of ASPIN Employing Trust-Region Control Strategies -- Convergence Analysis and Numerical Examples](https://mlsanity.com/paper/2104.05672/) (2021)
- [Nonlinear Schwarz preconditioning for nonlinear optimization problems with bound constraints](https://mlsanity.com/paper/2211.14780/) (2022)
- [A first-order multigrid method for bound-constrained convex optimization](https://mlsanity.com/paper/1602.03771/) (2016)
- [A note on solving nonlinear optimization problems in variable precision](https://mlsanity.com/paper/1812.03467/) (2018)
- [Recursive multilevel trust region method with application to fully monolithic phase-field models of brittle fracture](https://mlsanity.com/paper/1903.00379/) (2019)
- [Accelerated Additive Schwarz Methods for Convex Optimization with Adaptive Restart](https://mlsanity.com/paper/2011.02695/) (2020)
- [Trust-region algorithms: probabilistic complexity and intrinsic noise with applications to subsampling techniques](https://mlsanity.com/paper/2112.06176/) (2021)
- [Parallel Trust-Region Approaches in Neural Network Training: Beyond Traditional Methods](https://mlsanity.com/paper/2312.13677/) (2023)
- [Additive Schwarz Methods for Convex Optimization with Backtracking](https://mlsanity.com/paper/2110.07172/) (2021)
- [A multigrid solver for PDE-constrained optimization with uncertain inputs](https://mlsanity.com/paper/2302.13680/) (2023)
- [Complexity of Adagrad and other first-order methods for nonconvex optimization problems with bounds constraints](https://mlsanity.com/paper/2406.15793/) (2024)

## Built on by (most cited first, 3 of 3)

- [Leveraging Operator Learning to Accelerate Convergence of the Preconditioned Conjugate Gradient Method](https://mlsanity.com/paper/2508.00101/) (2025)
- [Multi-Preconditioned LBFGS for Training Finite-Basis PINNs](https://mlsanity.com/paper/2601.08709/) (2026)
- [Two-level domain-decomposition AdaGrad method for scalable training of graph neural networks](https://mlsanity.com/paper/2608.22575/) (2026)

## Abstract (arXiv)

Two OFFO (Objective-Function Free Optimization) noise tolerant algorithms are presented that handle bound constraints, inexact gradients and use second-order information when available.The first is a multi-level method exploiting a hierarchical description of the problem and the second is a domain-decomposition method covering the standard addditive Schwarz decompositions. Both are generalizations of the first-order AdaGrad algorithm for unconstrained optimization. Because these algorithms share a common theoretical framework, a single convergence/complexity theory is provided which covers them both. Its main result is that, with high probability, both methods need at most O(ε-2) iterations and noisy gradient evaluations to compute an ε-approximate first-order critical point of the bound-constrained problem. Extensive numerical experiments are discussed on applications ranging from PDE-based problems to deep neural network training, illustrating their remarkable computational efficiency.