# What Representational Similarity Measures Imply about Decodable Information

arXiv: https://arxiv.org/abs/2411.08197 · HTML: https://mlsanity.com/paper/2411.08197/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 38 references that are in the map.
- First posted on arXiv 12 Nov 2024.

## Builds on (most cited first, 17 of 17)

- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Getting aligned on representational alignment](https://mlsanity.com/paper/2310.13018/) (2023)
- [Similarity of Neural Network Models: A Survey of Functional and Representational Measures](https://mlsanity.com/paper/2305.06329/) (2023)
- [Generalized Shape Metrics on Neural Representations](https://mlsanity.com/paper/2110.14739/) (2021)
- [Reliability of CKA as a Similarity Measure in Deep Learning](https://mlsanity.com/paper/2210.16156/) (2022)
- [Universality and individuality in neural dynamics across large populations of recurrent networks](https://mlsanity.com/paper/1907.08549/) (2019)
- [Neural tuning and representational geometry](https://mlsanity.com/paper/2104.09743/) (2021)
- [Learned feature representations are biased by complexity, learning order, position, and more](https://mlsanity.com/paper/2405.05847/) (2024)
- [Interpreting Encoding and Decoding Models](https://mlsanity.com/paper/1812.00278/) (2018)
- [ReSi: A Comprehensive Benchmark for Representational Similarity Measures](https://mlsanity.com/paper/2408.00531/) (2024)
- [Soft Matching Distance: A metric on neural representations that captures single-neuron tuning](https://mlsanity.com/paper/2311.09466/) (2023)
- [GULP: a prediction-based metric between representations](https://mlsanity.com/paper/2210.06545/) (2022)
- [On Simplicity and Complexity in the Brave New World of Large-Scale Neuroscience](https://mlsanity.com/paper/1503.08779/) (2015)
- [Duality of Bures and Shape Distances with Implications for Comparing Neural Representations](https://mlsanity.com/paper/2311.11436/) (2023)
- [Differentiable Optimization of Similarity Scores Between Models and Brains](https://mlsanity.com/paper/2407.07059/) (2024)
- [Estimating Shape Distances on Neural Representations with Limited Samples](https://mlsanity.com/paper/2310.05742/) (2023)

## Built on by (most cited first, 18 of 18)

- [Revisiting the Platonic Representation Hypothesis: An Aristotelian View](https://mlsanity.com/paper/2602.14486/) (2026)
- [Representation Potentials of Foundation Models for Multimodal Alignment: A Survey](https://mlsanity.com/paper/2510.05184/) (2025)
- [Representation biases: will we achieve complete understanding by analyzing representations?](https://mlsanity.com/paper/2507.22216/) (2025)
- [Connecting Neural Models Latent Geometries with Relative Geodesic Representations](https://mlsanity.com/paper/2506.01599/) (2025)
- [Geometry-aware similarity metrics for neural representations on Riemannian and statistical manifolds](https://mlsanity.com/paper/2603.28764/) (2026)
- [When Embedding Models Meet: Procrustes Bounds and Applications](https://mlsanity.com/paper/2510.13406/) (2025)
- [Bayesian Comparisons Between Representations](https://mlsanity.com/paper/2411.08739/) (2024)
- [Can Biologically Plausible Temporal Credit Assignment Rules Match BPTT for Neural Similarity? E-prop as an Example](https://mlsanity.com/paper/2506.06904/) (2025)
- [How Data Augmentation Shapes Neural Representations](https://mlsanity.com/paper/2605.15306/) (2026)
- [Stimulus symmetries can confound representational similarity analyses](https://mlsanity.com/paper/2605.21324/) (2026)
- [Quantifying task-relevant representational similarity using decision variable correlation](https://mlsanity.com/paper/2506.02164/) (2025)
- [Knowledge distillation through geometry-aware representational alignment](https://mlsanity.com/paper/2509.25253/) (2025)
- [Estimating Dimensionality of Neural Representations from Finite Samples](https://mlsanity.com/paper/2509.26560/) (2025)
- [Do Reasoning Models Enhance Embedding Models?](https://mlsanity.com/paper/2601.21192/) (2026)
- [Beyond Activation Alignment: The Geometry of Neural Sensitivity](https://mlsanity.com/paper/2605.03222/) (2026)
- [PRISM: A Geometric Risk Bound for Decomposing Drift into Scale, Shape, and Head](https://mlsanity.com/paper/2605.11608/) (2026)
- [The Cross-Architecture Substrate: A Domain-Transcendent, Calibration-Surviving Geometric Invariant of Modern Vision Encoders](https://mlsanity.com/paper/2606.07882/) (2026)
- [Decoder-Preserving Sparse Autoencoders: Which Readouts Survive Sparse Compression?](https://mlsanity.com/paper/2607.17425/) (2026)

## Abstract (arXiv)

Neural responses encode information that is useful for a variety of downstream tasks. A common approach to understand these systems is to build regression models or ``decoders'' that reconstruct features of the stimulus from neural responses. Popular neural network similarity measures like centered kernel alignment (CKA), canonical correlation analysis (CCA), and Procrustes shape distance, do not explicitly leverage this perspective and instead highlight geometric invariances to orthogonal or affine transformations when comparing representations. Here, we show that many of these measures can, in fact, be equivalently motivated from a decoding perspective. Specifically, measures like CKA and CCA quantify the average alignment between optimal linear readouts across a distribution of decoding tasks. We also show that the Procrustes shape distance upper bounds the distance between optimal linear readouts and that the converse holds for representations with low participation ratio. Overall, our work demonstrates a tight link between the geometry of neural representations and the ability to linearly decode information. This perspective suggests new ways of measuring similarity between neural systems and also provides novel, unifying interpretations of existing measures.