# Higher-order Comparisons of Sentence Encoder Representations

arXiv: https://arxiv.org/abs/1909.00303 · HTML: https://mlsanity.com/paper/1909.00303/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Sep 2019.

## Built on by (most cited first, 10 of 10)

- [Do Transformer Models Show Similar Attention Patterns to Task-Specific Human Gaze?](https://mlsanity.com/paper/2205.10226/) (2022)
- [A comparative evaluation and analysis of three generations of Distributional Semantic Models](https://mlsanity.com/paper/2105.09825/) (2021)
- [Do Large Language Models Mirror Cognitive Language Processing?](https://mlsanity.com/paper/2402.18023/) (2024)
- [How Familiar Does That Sound? Cross-Lingual Representational Similarity Analysis of Acoustic Word Embeddings](https://mlsanity.com/paper/2109.10179/) (2021)
- [The Copenhagen Corpus of Eye Tracking Recordings from Natural Reading of Danish Texts](https://mlsanity.com/paper/2204.13311/) (2022)
- [Decoding In-Context Learning: Neuroscience-inspired Analysis of Representations in Large Language Models](https://mlsanity.com/paper/2310.00313/) (2023)
- [Linguistic Interpretability of Transformer-based Language Models: a systematic review](https://mlsanity.com/paper/2504.08001/) (2025)
- [Discrete representations in neural models of spoken language](https://mlsanity.com/paper/2105.05582/) (2021)
- [The Grammar of Transformers: A Systematic Review of Interpretability Research on Syntactic Knowledge in Language Models](https://mlsanity.com/paper/2601.19926/) (2026)
- [Disentangling the Linguistic Competence of Privacy-Preserving BERT](https://mlsanity.com/paper/2310.11363/) (2023)

## Abstract (arXiv)

Representational Similarity Analysis (RSA) is a technique developed by neuroscientists for comparing activity patterns of different measurement modalities (e.g., fMRI, electrophysiology, behavior). As a framework, RSA has several advantages over existing approaches to interpretation of language encoders based on probing or diagnostic classification: namely, it does not require large training samples, is not prone to overfitting, and it enables a more transparent comparison between the representational geometries of different models and modalities. We demonstrate the utility of RSA by establishing a previously unknown correspondence between widely-employed pretrained language encoders and human processing difficulty via eye-tracking data, showcasing its potential in the interpretability toolbox for neural models