Causal Optimizer Interaction Calculus: Hidden Geometric Relaxation and Identifiable Interventions
Not on the map. Why?
Abstract (arXiv)
Optimizer experiments observe responses to algorithmic configurations without uniquely revealing hidden mechanisms. We develop a causal optimizer interaction calculus that separates pathwise realization, Mobius decomposition, and experimental identification. Under a fixed innovation coupling, every finite-horizon innovation-driven optimizer admits a behaviorally minimal pathwise realization. For any finite effect support and intervention design, an incidence operator gives the complete observational gauge, exact identifiability, sharp quotient stability, held-out predictions, and exact noiseless configuration complexity. Smooth hidden relaxation generates interactions through inverse hidden-state stiffness. Building on this structural law, we prove an observable-readout transfer theorem: arbitrary smooth update or trace readouts inherit an explicit five-term interaction through first and second hidden responses. Unlike the reduced optimal value, a general readout has no universal interaction sign. Its Boolean effects remain exact integrals of continuous interaction curvature and can therefore be identified by factorial interventions. We also derive Gaussian quotient minimax risk, exact confidence sets and tests, misspecification decomposition, certified downstream decisions, and optimal replication. A controlled real-data experiment on a 65-dimensional strongly convex logistic model validates the complete reduced-value chain. Boolean effects and independently integrated curvature agree within 4.21e-11, while nine held-out continuous intensities agree within 8.88e-13. Gaussian campaigns attain the predicted coverage and power, and 4,500 real-minibatch observations reject an order-two interaction model. Neural trace audits provide complementary evidence that the declared response classes remain informative in nonconvex training.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 47 references that are in the map. First posted on arXiv 8 Jul 2026.
Builds on
18 of 47 references.
References · 18
- 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 905
- 2015Optimizing Neural Networks with Kronecker-factored Approximate Curvaturecited by 796
- 2016Learning to learn by gradient descent by gradient descentcited by 620
- 2017OptNet: Differentiable Optimization as a Layer in Neural Networkscited by 599
- 2018Shampoo: Preconditioned Stochastic Tensor Optimizationcited by 426
- 2018Bilevel Programming for Hyperparameter Optimization and Meta-Learningcited by 405
- 2023Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-trainingcited by 265
- 2015A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insightscited by 206
- 2021Efficient and Modular Implicit Differentiationcited by 203
- 2024SOAP: Improving and Stabilizing Shampoo using Adamcited by 192
- 2019The Shapley Taylor Interaction Indexcited by 143
- 2018Automated Algorithm Selection: Survey and Perspectivescited by 114
6 more
- 2014Analysis and Design of Optimization Algorithms via Integral Quadratic Constraintscited by 105
- 2020Explaining Explanations: Axiomatic Feature Interactions for Deep Networkscited by 103
- 2022VeLO: Training Versatile Learned Optimizers by Scaling Upcited by 57
- 2015Smooth Strongly Convex Interpolation and Exact Worst-case Performance of First-order Methodscited by 37
- 2025An Exploration of Non-Euclidean Gradient Descent: Muon and its Many Variantscited by 28
- 2026Optimization Geometrodynamics: Variational Reduction and Interaction Curvaturecited by 1
Built on by
Not cited yet.