# Estimating Shape Distances on Neural Representations with Limited Samples

arXiv: https://arxiv.org/abs/2310.05742 · HTML: https://mlsanity.com/paper/2310.05742/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 44 references that are in the map.
- First posted on arXiv 9 Oct 2023.

## Builds on (most cited first, 16 of 16)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [An Introduction to Matrix Concentration Inequalities](https://mlsanity.com/paper/1501.01571/) (2015)
- [Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depth](https://mlsanity.com/paper/2010.15327/) (2020)
- [Similarity of Neural Network Models: A Survey of Functional and Representational Measures](https://mlsanity.com/paper/2305.06329/) (2023)
- [Generalized Shape Metrics on Neural Representations](https://mlsanity.com/paper/2110.14739/) (2021)
- [Universality and individuality in neural dynamics across large populations of recurrent networks](https://mlsanity.com/paper/1907.08549/) (2019)
- [Recurrence is required to capture the representational dynamics of the human visual system](https://mlsanity.com/paper/1903.05946/) (2019)
- [Representational dissimilarity metric spaces for stochastic neural networks](https://mlsanity.com/paper/2211.11665/) (2022)
- [Estimating the Spectral Density of Large Implicit Matrices](https://mlsanity.com/paper/1802.03451/) (2018)
- [Statistical inference on representational geometries](https://mlsanity.com/paper/2112.09200/) (2021)
- [Spectrum Estimation from Samples](https://mlsanity.com/paper/1602.00061/) (2016)
- [Neural Networks as Paths through the Space of Representations](https://mlsanity.com/paper/2206.10999/) (2022)
- [Statistical Neuroscience in the Single Trial Limit](https://mlsanity.com/paper/2103.05075/) (2021)

## Built on by (most cited first, 4 of 4)

- [What Representational Similarity Measures Imply about Decodable Information](https://mlsanity.com/paper/2411.08197/) (2024)
- [Spectral Analysis of Representational Similarity with Limited Neurons](https://mlsanity.com/paper/2502.19648/) (2025)
- [Can Biologically Plausible Temporal Credit Assignment Rules Match BPTT for Neural Similarity? E-prop as an Example](https://mlsanity.com/paper/2506.06904/) (2025)
- [Stimulus symmetries can confound representational similarity analyses](https://mlsanity.com/paper/2605.21324/) (2026)

## Abstract (arXiv)

Measuring geometric similarity between high-dimensional network representations is a topic of longstanding interest to neuroscience and deep learning. Although many methods have been proposed, only a few works have rigorously analyzed their statistical efficiency or quantified estimator uncertainty in data-limited regimes. Here, we derive upper and lower bounds on the worst-case convergence of standard estimators of shape distanceunicodex2014a measure of representational dissimilarity proposed by Williams et al. (2021).These bounds reveal the challenging nature of the problem in high-dimensional feature spaces. To overcome these challenges, we introduce a new method-of-moments estimator with a tunable bias-variance tradeoff. We show that this estimator achieves substantially lower bias than standard estimators in simulation and on neural data, particularly in high-dimensional settings. Thus, we lay the foundation for a rigorous statistical theory for high-dimensional shape analysis, and we contribute a new estimation method that is well-suited to practical scientific settings.