Representation Geometry as a Diagnostic for Out-of-Distribution Robustness
Not on the map. Why?
Abstract (arXiv)
Robust generalization under distribution shift remains difficult to monitor and optimize in the absence of target-domain labels, as models with similar in-distribution accuracy can exhibit markedly different out-of-distribution (OOD) performance. While prior work has focused on training-time regularization and low-order representation statistics, little is known about whether the geometric structure of learned embeddings provides reliable post-hoc signals of robustness. We propose a geometry-based diagnostic framework that constructs class-conditional mutual k-nearest-neighbor graphs from in-distribution embeddings and extracts two complementary invariants: a global spectral complexity proxy based on the reduced log-determinant of the normalized Laplacian, and a local smoothness measure based on Ollivier--Ricci curvature. Across multiple architectures, training regimes, and corruption benchmarks, we find that lower spectral complexity and higher mean curvature consistently predict stronger OOD accuracy across checkpoints. Controlled perturbations and topological analyses further show that these signals reflect meaningful representation structure rather than superficial embedding statistics. Our results demonstrate that representation geometry enables interpretable, label-free robustness diagnosis and supports reliable unsupervised checkpoint selection under distribution shift.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 14 of its 22 references that are in the map. First posted on arXiv 3 Feb 2026.
Builds on
14 of 22 references.
References · 14
- 2019Invariant Risk Minimizationcited by 1,832
- 2019Similarity of Neural Network Representations Revisitedcited by 1,695
- 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
- 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,393
- 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,194
- 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 737
- 2019Fantastic Generalization Measures and Where to Find Themcited by 507
- 2021Understanding over-squashing and bottlenecks on graphs via curvaturecited by 471
- 2021SWAD: Domain Generalization by Seeking Flat Minimacited by 431
- 2020Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depthcited by 241
- 2020Invariant Risk Minimization Gamescited by 188
- 2018Neural Persistence: A Complexity Measure for Deep Neural Networks Using Algebraic Topologycited by 73
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026Analytic Torsion and Spectral Gap Capture Persistent-Laplacian Performancecited by 0
- 2026STAMP: Predicting Out-of-Distribution Generalization without Target Datacited by 0
- 2026Pretraining Shapes Spectral Structure: Architecture- and Strategy-Conditional Prediction of OOD Robustness in Foundation Modelscited by 0
- 2026Pretraining Shapes Spectral Structure: Architecture- and Strategy-Conditional Prediction of OOD Robustness in Foundation Models7 Oct 2026
- 2026STAMP: Predicting Out-of-Distribution Generalization without Target Data26 Sep 2026
- 2026Analytic Torsion and Spectral Gap Capture Persistent-Laplacian Performance15 Jun 2026