# Grounding Representation Similarity with Statistical Testing

arXiv: https://arxiv.org/abs/2108.01661 · HTML: https://mlsanity.com/paper/2108.01661/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 42 references that are in the map.
- First posted on arXiv 3 Aug 2021.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning](https://mlsanity.com/paper/1712.05526/) (2017)
- [Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference](https://mlsanity.com/paper/1902.01007/) (2019)
- [Underspecification Presents Challenges for Credibility in Modern Machine Learning](https://mlsanity.com/paper/2011.03395/) (2020)
- [Word Translation Without Parallel Data](https://mlsanity.com/paper/1710.04087/) (2017)
- [What do you learn from context? Probing for sentence structure in contextualized word representations](https://mlsanity.com/paper/1905.06316/) (2019)
- [Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML](https://mlsanity.com/paper/1909.09157/) (2019)
- [Weight Poisoning Attacks on Pre-trained Models](https://mlsanity.com/paper/2004.06660/) (2020)
- [Insights on representational similarity in neural networks with canonical correlation](https://mlsanity.com/paper/1806.05759/) (2018)
- [Understanding image representations by measuring their equivariance and equivalence](https://mlsanity.com/paper/1411.5908/) (2014)
- [Convergent Learning: Do different neural networks learn the same representations?](https://mlsanity.com/paper/1511.07543/) (2015)
- [Stress Test Evaluation for Natural Language Inference](https://mlsanity.com/paper/1806.00692/) (2018)
- [What do Neural Machine Translation Models Learn about Morphology?](https://mlsanity.com/paper/1704.03471/) (2017)
- [Dissecting Contextual Word Embeddings: Architecture and Representation](https://mlsanity.com/paper/1808.08949/) (2018)
- [Offline bilingual word vectors, orthogonal transformations and the inverted softmax](https://mlsanity.com/paper/1702.03859/) (2017)
- [BERTs of a feather do not generalize together: Large variability in generalization across models with similar test set performance](https://mlsanity.com/paper/1911.02969/) (2019)
- [Towards Understanding Learning Representations: To What Extent Do Different Neural Networks Learn the Same Representation](https://mlsanity.com/paper/1810.11750/) (2018)
- [Towards falsifiable interpretability research](https://mlsanity.com/paper/2010.12016/) (2020)
- [Are Larger Pretrained Language Models Uniformly Better? Comparing Performance at the Instance Level](https://mlsanity.com/paper/2105.06020/) (2021)
- [Investigating Transferability in Pretrained Language Models](https://mlsanity.com/paper/2004.14975/) (2020)
- [Deep Learning Backdoors](https://mlsanity.com/paper/2007.08273/) (2020)
- [Knowledge Consistency between Neural Networks and Beyond](https://mlsanity.com/paper/1908.01581/) (2019)

## Built on by (most cited first, 29 of 29)

- [Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks](https://mlsanity.com/paper/2207.13243/) (2022)
- [Universal Neurons in GPT2 Language Models](https://mlsanity.com/paper/2401.12181/) (2024)
- [On the Symmetries of Deep Learning Models and their Internal Representations](https://mlsanity.com/paper/2205.14258/) (2022)
- [Towards Universality: Studying Mechanistic Similarity Across Language Model Architectures](https://mlsanity.com/paper/2410.06672/) (2024)
- [ModelDiff: A Framework for Comparing Learning Algorithms](https://mlsanity.com/paper/2211.12491/) (2022)
- [A Closer Look at Model Adaptation using Feature Distortion and Simplicity Bias](https://mlsanity.com/paper/2303.13500/) (2023)
- [Deconfounded Representation Similarity for Comparison of Neural Networks](https://mlsanity.com/paper/2202.00095/) (2022)
- [A Spectral Theory of Neural Prediction and Alignment](https://mlsanity.com/paper/2309.12821/) (2023)
- [Measuring Representational Robustness of Neural Networks Through Shared Invariances](https://mlsanity.com/paper/2206.11939/) (2022)
- [The Curse of Low Task Diversity: On the Failure of Transfer Learning to Outperform MAML and Their Empirical Equivalence](https://mlsanity.com/paper/2208.01545/) (2022)
- [Model Stitching: Looking For Functional Similarity Between Representations](https://mlsanity.com/paper/2303.11277/) (2023)
- [System identification of neural systems: If we got it right, would we know?](https://mlsanity.com/paper/2302.06677/) (2023)
- [Auditing Visualizations: Transparency Methods Struggle to Detect Anomalous Behavior](https://mlsanity.com/paper/2206.13498/) (2022)
- [ContraSim -- Analyzing Neural Representations Based on Contrastive Learning](https://mlsanity.com/paper/2303.16992/) (2023)
- [Can Generative Models Improve Self-Supervised Representation Learning?](https://mlsanity.com/paper/2403.05966/) (2024)
- [On Affine Homotopy between Language Encoders](https://mlsanity.com/paper/2406.02329/) (2024)
- [Do Invariances in Deep Neural Networks Align with Human Perception?](https://mlsanity.com/paper/2111.14726/) (2021)
- [Does MAML Only Work via Feature Re-use? A Data Centric Perspective](https://mlsanity.com/paper/2112.13137/) (2021)
- [What do CNNs Learn in the First Layer and Why? A Linear Systems Perspective](https://mlsanity.com/paper/2206.02454/) (2022)
- [GIST: Generated Inputs Sets Transferability in Deep Learning](https://mlsanity.com/paper/2311.00801/) (2023)
- [The Curse of Zero Task Diversity: On the Failure of Transfer Learning to Outperform MAML and their Empirical Equivalence](https://mlsanity.com/paper/2112.13121/) (2021)
- [Has Your Pretrained Model Improved? A Multi-head Posterior Based Approach](https://mlsanity.com/paper/2401.02987/) (2024)
- [Identifying Sub-networks in Neural Networks via Functionally Similar Representations](https://mlsanity.com/paper/2410.16484/) (2024)
- [PRISM: A Geometric Risk Bound for Decomposing Drift into Scale, Shape, and Head](https://mlsanity.com/paper/2605.11608/) (2026)
- [Statevector-to-Hardware Reconstruction of a Four-Qubit ZZ Quantum Kernel: A Single-Backend Case Study of Three Execution Jobs](https://mlsanity.com/paper/2607.20377/) (2026)
- [Reversible Unlearnable Examples: Towards the Copyright Protection in Deep Learning Era](https://mlsanity.com/paper/2608.06211/) (2026)
- [Perplexity Cost Understates What Activation Quantisation Breaks](https://mlsanity.com/paper/2609.23125/) (2026)
- [Hidden Activations are not Enough I: Knowledge Matrices as Higher Representations](https://mlsanity.com/paper/2609.34166/) (2026)
- [Representational and Functional Robustness to Electrode Montages in EEG Foundation Models](https://mlsanity.com/paper/2609.36288/) (2026)

## Abstract (arXiv)

To understand neural network behavior, recent works quantitatively compare different networks' learned representations using canonical correlation analysis (CCA), centered kernel alignment (CKA), and other dissimilarity measures. Unfortunately, these widely used measures often disagree on fundamental observations, such as whether deep networks differing only in random initialization learn similar representations. These disagreements raise the question: which, if any, of these dissimilarity measures should we believe? We provide a framework to ground this question through a concrete test: measures should have sensitivity to changes that affect functional behavior, and specificity against changes that do not. We quantify this through a variety of functional behaviors including probing accuracy and robustness to distribution shift, and examine changes such as varying random initialization and deleting principal components. We find that current metrics exhibit different weaknesses, note that a classical baseline performs surprisingly well, and highlight settings where all metrics appear to fail, thus providing a challenge set for further improvement.