When Descent Is Too Stable: Event-Triggered Hamiltonian Learning to Optimize
Not on the map. Why?
Abstract (arXiv)
Fixed-budget nonconvex optimization can fail not because local descent is unstable, but because it is too stable: after reaching a nearby stationary point, an optimizer may spend the remaining evaluations refining an uninformative local minimum. We formulate this failure mode as a control problem over optimizer dynamics, where the learner must decide when to descend, when to exploit a promising basin, and when stagnation should trigger movement elsewhere. We introduce SHAPE, a structured adaptive port-Hamiltonian task-family optimizer for event-triggered minima hunting under local information. Starting from gradient-descent dynamics, SHAPE lifts optimization to an augmented phase space (q, p), where the primal state q represents the candidate solution, the cotangent variable p carries directional sensitivity, and a controller u provides processed information from current gradient oracle. Within each stage, a learned Hamiltonian vector field induces structured local descent; across stages, a fixed event clock in the implementation updates ports and memory when local equilibria are detected, with stage-dependent horizons treated in the analysis as a direct generalization. This design preserves a passivity-compatible structure while allowing the same trained policy to use clean, stochastic, or estimated gradient inputs. Experiments on fixed-budget nonconvex optimization tasks show that SHAPE improves best-so-far performance compared with fixed-policy optimizers. These results suggest that adaptive Hamiltonian energy shaping provides a principled mechanism for balancing descent, exploration, and budget allocation in difficult optimization landscapes.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 23 of its 52 references that are in the map. First posted on arXiv 7 May 2026.
Builds on
23 of 52 references.
References · 23
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
- 2017Visualizing the Loss Landscape of Neural Netscited by 1,130
- 2018Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizoncited by 623
- 2016Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Conditioncited by 539
- 2018Shampoo: Preconditioned Stochastic Tensor Optimizationcited by 426
- 2014Stochastic Gradient Hamiltonian Monte Carlocited by 343
- 2019Lookahead Optimizer: k steps forward, 1 step backcited by 336
- 2023Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-trainingcited by 265
- 2021Learning to Optimize: A Primer and A Benchmarkcited by 208
- 2024SOAP: Improving and Stabilizing Shampoo using Adamcited by 192
11 more
- 2016A Variational Perspective on Accelerated Methods in Optimizationcited by 110
- 2023Lion Secretly Solves Constrained Optimization: As Lyapunov Predictscited by 51
- 2022Model-Based Deep Learning: On the Intersection of Deep Learning and Optimizationcited by 50
- 2022Practical tradeoffs between memory, compute, and performance in learned optimizerscited by 32
- 2019Conformal Symplectic and Relativistic Optimizationcited by 29
- 2023Towards Constituting Mathematical Structures for Learning to Optimizecited by 13
- 2019Stochastic Gradient Hamiltonian Monte Carlo for Non-Convex Learningcited by 8
- 2025Deep FlexQP: Accelerated Nonlinear Programming via Deep Unfoldingcited by 4
- 2025Hamiltonian Descent Algorithms for Optimization: Accelerated Rates via Randomized Integration Timecited by 2
- 2024Non-asymptotic convergence analysis of the stochastic gradient Hamiltonian Monte Carlo algorithm with discontinuous stochastic gradient with applications to training of ReLU neural networkscited by 1
- 2024Port-Hamiltonian nonlinear systemscited by 1
Built on by
Not cited yet.