# Information-Induced Training Geometry: Exact Reduction, Canonical Completion, and Structured Expressivity

arXiv: https://arxiv.org/abs/2609.12991 · HTML: https://mlsanity.com/paper/2609.12991/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 33 references that are in the map.
- First posted on arXiv 11 Jul 2026.

## Builds on (most cited first, 9 of 9)

- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Optimizing Neural Networks with Kronecker-factored Approximate Curvature](https://mlsanity.com/paper/1503.05671/) (2015)
- [New insights and perspectives on the natural gradient method](https://mlsanity.com/paper/1412.1193/) (2014)
- [Shampoo: Preconditioned Stochastic Tensor Optimization](https://mlsanity.com/paper/1802.09568/) (2018)
- [An Introduction to Matrix Concentration Inequalities](https://mlsanity.com/paper/1501.01571/) (2015)
- [Limitations of the Empirical Fisher Approximation for Natural Gradient Descent](https://mlsanity.com/paper/1905.12558/) (2019)
- [Fast Approximate Natural Gradient Descent in a Kronecker-factored Eigenbasis](https://mlsanity.com/paper/1806.03884/) (2018)
- [A New Perspective on Shampoo's Preconditioner](https://mlsanity.com/paper/2406.17748/) (2024)
- [The Information Geometry of Mirror Descent](https://mlsanity.com/paper/1310.7780/) (2013)

## Abstract (arXiv)

Training data constrains optimizer geometry through the covectors visible to a declared information channel. We study how such partial information determines a full positive cometric relative to a reference and which degrees of freedom remain unidentified. Our central result resolves full-column-rank positive-definite compression under affine-invariant Riemannian geometry. The compression map is a split-Hadamard metric submetry and admits an explicit unique completion that is the affine-invariant nearest full geometry realizing a visible target and yields exact full-to-visible variational reduction. When the channel moves, the completions form a gauge-invariant rank stratification of the positive-definite cone. Its closed-form pullback pair metric separates visible-metric motion from subspace rotation through a reference-mismatch weight, yields an explicit positive-semidefinite multi-direction Gram matrix, and exposes the precise singularity of reference-valued modes. The mechanism is explained by a metric theorem equating ball submetry, attained fiber distance, and lossless reduction of every monotone radial visible decision problem. A smooth split-Hadamard theorem supplies coherent information sheets, proximal commutation, and solution-wise gradient-flow lifting. The positive-definite realization also gives closed-form prior-data shrinkage. Diagonal and block optimizer families reduce to relative-interior conic image tests with valid facial certificates, while deterministic and finite-sample bounds quantify recovery of the visible geometry and its subspace. Together these results characterize exact reduction, reference-dependent completion, and structured expressivity for the stated finite-dimensional affine-invariant model.