mlsanityData to 14 Oct 2026

Duality of Bures and Shape Distances with Implications for Comparing Neural Representations

arXiv 2311.11436 ↗First posted 19 Nov 2023stat.ML cs.LGcited by 15

Not on the map. Why?

Abstract (arXiv)

A multitude of (dis)similarity measures between neural network representations have been proposed, resulting in a fragmented research landscape. Most of these measures fall into one of two categories. First, measures such as linear regression, canonical correlations analysis (CCA), and shape distances, all learn explicit mappings between neural units to quantify similarity while accounting for expected invariances. Second, measures such as representational similarity analysis (RSA), centered kernel alignment (CKA), and normalized Bures similarity (NBS) all quantify similarity in summary statistics, such as stimulus-by-stimulus kernel matrices, which are already invariant to expected symmetries. Here, we take steps towards unifying these two broad categories of methods by observing that the cosine of the Riemannian shape distance (from category 1) is equal to NBS (from category 2). We explore how this connection leads to new interpretations of shape distances and NBS, and draw contrasts of these measures with CKA, a popular similarity measure in the deep learning literature.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 19 of its 57 references that are in the map. First posted on arXiv 19 Nov 2023.

Builds on

19 of 57 references.

References · 19

  1. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  2. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  3. 2017Deep Neural Networks as Gaussian Processescited by 622
  4. 2018Insights on representational similarity in neural networks with canonical correlationcited by 324
  5. 2018Gaussian Process Behaviour in Wide Deep Neural Networkscited by 317
  6. 2020Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depthcited by 245
  7. 2015Convergent Learning: Do different neural networks learn the same representations?cited by 220
  8. 2023Getting aligned on representational alignmentcited by 143
  9. 2023Similarity of Neural Network Models: A Survey of Functional and Representational Measurescited by 131
  10. 2021Generalized Shape Metrics on Neural Representationscited by 112
  11. 2022Reliability of CKA as a Similarity Measure in Deep Learningcited by 87
  12. 2019Universality and individuality in neural dynamics across large populations of recurrent networkscited by 61
7 more
  1. 2019Recurrence is required to capture the representational dynamics of the human visual systemcited by 58
  2. 2018Generalizing Point Embeddings using the Wasserstein Space of Elliptical Distributionscited by 20
  3. 2018Procrustes Metrics on Covariance Operators and Optimal Transportation of Gaussian Processescited by 19
  4. 2022Representational dissimilarity metric spaces for stochastic neural networkscited by 17
  5. 2020Similarity of Neural Networks with Gradientscited by 16
  6. 2021Statistical inference on representational geometriescited by 13
  7. 2016Inferring brain-computational mechanisms with models of activity measurementscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 6; 2026: 4.

15 papers.

Sort:

By area:

  1. 2025From Language to Cognition: How LLMs Outgrow the Human Language Networkcited by 30
  2. 2026There Will Be a Scientific Theory of Deep Learningcited by 29
  3. 2024What Representational Similarity Measures Imply about Decodable Informationcited by 18
  4. 2024Differentiable Optimization of Similarity Scores Between Models and Brainscited by 10
  5. 2024On the universality of neural encodings in CNNscited by 4
  6. 2024Brain-Like Language Processing via a Shallow Untrained Multihead Attention Networkcited by 3
  7. 2025Model-brain comparison using inter-animal transformscited by 3
  8. 2024Bayesian Comparisons Between Representationscited by 1
  9. 2025Can Biologically Plausible Temporal Credit Assignment Rules Match BPTT for Neural Similarity? E-prop as an Examplecited by 1
  10. 2026Modulating Cross-Modal Convergence with Single-Stimulus, Intra-Modal Dispersioncited by 1
  11. 2026How Data Augmentation Shapes Neural Representationscited by 1
  12. 2025Supervised Quadratic Feature Analysis: Information Geometry Approach for Dimensionality Reductioncited by 0
3 more
  1. 2025Knowledge distillation through geometry-aware representational alignmentcited by 0
  2. 2025Sparse components distinguish visual pathways & their alignment to neural networkscited by 0
  3. 2026Scale, Drivers, and Limits of Representational Alignment Across Large Language Modelscited by 0