# Gauge-invariant representation holonomy

arXiv: https://arxiv.org/abs/2601.21653 · HTML: https://mlsanity.com/paper/2601.21653/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 18 references that are in the map.
- First posted on arXiv 29 Jan 2026.

## Builds on (most cited first, 10 of 10)

- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [Insights on representational similarity in neural networks with canonical correlation](https://mlsanity.com/paper/1806.05759/) (2018)
- [Gauge Equivariant Convolutional Networks and the Icosahedral CNN](https://mlsanity.com/paper/1902.04615/) (2019)
- [Understanding image representations by measuring their equivariance and equivalence](https://mlsanity.com/paper/1411.5908/) (2014)
- [A Riemannian Network for SPD Matrix Learning](https://mlsanity.com/paper/1608.04233/) (2016)
- [Optimal whitening and decorrelation](https://mlsanity.com/paper/1512.00809/) (2015)
- [The Lie Derivative for Measuring Learned Equivariance](https://mlsanity.com/paper/2210.02984/) (2022)
- [Parallel Transport Convolution: A New Tool for Convolutional Neural Networks on Manifolds](https://mlsanity.com/paper/1805.07857/) (2018)

## Built on by (most cited first, 7 of 7)

- [A Gauge Theory of Superposition: Toward a Sheaf-Theoretic Atlas of Neural Representations](https://mlsanity.com/paper/2603.00824/) (2026)
- [A Fiber Criterion for Representation Identifiability in Supervised Learning](https://mlsanity.com/paper/2606.01092/) (2026)
- [Bayes-Sufficient Representations in Supervised Learning](https://mlsanity.com/paper/2606.04045/) (2026)
- [Do Active SAE Feature Planes Carry More Holonomy? A Preregistered Reversal in Gemma](https://mlsanity.com/paper/2607.20522/) (2026)
- [Do Sheaf Neural Networks Use Holonomy? A Measure--Intervene--Control Study](https://mlsanity.com/paper/2607.19514/) (2026)
- [Understanding Differentiable Embeddings Through Differential and Integral Geometry](https://mlsanity.com/paper/2608.06809/) (2026)
- [Activation-Space Order-Swap Geometry: A Site-Asymmetry Audit](https://mlsanity.com/paper/2608.25315/) (2026)

## Abstract (arXiv)

Deep networks learn internal representations whose geometry--how features bend, rotate, and evolve--affects both generalization and robustness. Existing similarity measures such as CKA or SVCCA capture pointwise overlap between activation sets, but miss how representations change along input paths. Two models may appear nearly identical under these metrics yet respond very differently to perturbations or adversarial stress. We introduce representation holonomy, a gauge-invariant statistic that measures this path dependence. Conceptually, holonomy quantifies the "twist" accumulated when features are parallel-transported around a small loop in input space: flat representations yield zero holonomy, while nonzero values reveal hidden curvature. Our estimator fixes gauge through global whitening, aligns neighborhoods using shared subspaces and rotation-only Procrustes, and embeds the result back to the full feature space. We prove invariance to orthogonal (and affine, post-whitening) transformations, establish a linear null for affine layers, and show that holonomy vanishes at small radii. Empirically, holonomy increases with loop radius, separates models that appear similar under CKA, and correlates with adversarial and corruption robustness. It also tracks training dynamics as features form and stabilize. Together, these results position representation holonomy as a practical and scalable diagnostic for probing the geometric structure of learned representations beyond pointwise similarity.