A Non-Monotone Preconditioned Trust-Region Method for Neural Network Training
Not on the map. Why?
Abstract (arXiv)
Training deep neural networks at scale can benefit from domain decomposition, where the network is split into subdomains trained in parallel and coupled by a global trust-region mechanism. Building on the Additively Preconditioned Trust-Region Strategy (APTS), we propose a non-monotone variant with a nonlinear additive Schwarz preconditioner that combines parallel subdomain corrections with global coarse-space directions. A windowed acceptance criterion allows controlled objective increases, avoiding needless rejection of effective coarse steps. The resulting non-monotone APTS (NAPTS) preserves accuracy while reducing CPU time by 30% and cutting rejected steps to one third of those in APTS.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 8 of its 15 references that are in the map. First posted on arXiv 14 May 2026.
Builds on
8 of 15 references.
References · 8
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2018Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysiscited by 154
- 2021Globally Convergent Multilevel Training of Deep Residual Networkscited by 11
- 2017A Stochastic Trust Region Algorithm Based on Careful Step Normalizationcited by 10
- 2021A Survey and Empirical Evaluation of Parallel Deep Learning Frameworkscited by 7
- 2023Parallel Trust-Region Approaches in Neural Network Training: Beyond Traditional Methodscited by 2
- 2025Data-Parallel Neural Network Training via Nonlinearly Preconditioned Trust-Region Methodcited by 2
- 2025An Additively Preconditioned Trust Region Strategy for Machine Learningcited by 1
Built on by
Not cited yet.