mlsanityData to 14 Oct 2026

Optimization Geometrodynamics: Variational Reduction and Interaction Curvature

arXiv 2607.06723 ↗First posted 7 Jul 2026math.OC cs.LG math.DGcited by 1

Not on the map. Why?

Abstract (arXiv)

Adaptive optimizers carry hidden states that change how visible gradients become parameter motion. We develop optimization geometrodynamics as a variational theory of this hidden geometry. Infimal pushforward eliminates all hidden states realizing the same action and composes across optimizer hierarchies. Under smooth nondegeneracy, it yields hidden susceptibility and the Schur-complement curvature seen after relaxation. For affine pre-reduction perturbations, the induced interaction curvature is the negative-semidefinite operator -G^*H-1G, whose mixed entries integrate to finite mechanism contrasts. Our main realization is the determinant-one affine-invariant SPD action map Pmapsto PA. We prove a global analytic bundle with closed totally geodesic fibers and a unique analytic nearest-controller section. A strongly convex fiber theorem and an explicit logarithmic action residual give a globally linearly convergent solver from every feasible initializer, together with nonasymptotic value, controller-distance, and residual bounds and observable posterior stopping certificates. A conditional inexact result propagates supplied rigorous residual-error and radius majorants. The dense spectral kernel is confined to an active subspace of dimension r≤ 2m, yielding an explicit spectral-arithmetic operation bound and a strict dimensional reduction when r<d. For nested shape-normalized quadratic actions, canonical multi-secant projections satisfy an exact CAT(0) Pythagorean decrease and recover the determinant-one inverse Hessian shape at the sharp rank threshold d-1, provided the scalar gauge cH=(det H)1/d is known. These results turn the action bundle into an exact iterative computation with posterior certificates and a finite-identification theory.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 9 of its 41 references that are in the map. First posted on arXiv 7 Jul 2026.

Builds on

9 of 41 references.

References · 9

  1. 2015Optimizing Neural Networks with Kronecker-factored Approximate Curvaturecited by 796
  2. 2018Shampoo: Preconditioned Stochastic Tensor Optimizationcited by 426
  3. 2024SOAP: Improving and Stabilizing Shampoo using Adamcited by 192
  4. 2016First-order Methods for Geodesically Convex Optimizationcited by 93
  5. 2024A New Perspective on Shampoo's Preconditionercited by 59
  6. 2025An Exploration of Non-Euclidean Gradient Descent: Muon and its Many Variantscited by 28
  7. 2023Scalable Approximate Optimal Diagonal Preconditioningcited by 9
  8. 2016Block BFGS Methodscited by 3
  9. 2025Optimal Preconditioning is a Geodesically Convex Optimization Problemcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.