# Metric Learning Encoding Models: A Multivariate Framework for Interpreting Neural Representations

arXiv: https://arxiv.org/abs/2402.11608 · HTML: https://mlsanity.com/paper/2402.11608/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 58 references that are in the map.
- First posted on arXiv 18 Feb 2024.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Probing Classifiers: Promises, Shortcomings, and Advances](https://mlsanity.com/paper/2102.12452/) (2021)
- [Distribution-Free Predictive Inference For Regression](https://mlsanity.com/paper/1604.04173/) (2016)
- [Designing and Interpreting Probes with Control Tasks](https://mlsanity.com/paper/1909.03368/) (2019)
- [What do you learn from context? Probing for sentence structure in contextualized word representations](https://mlsanity.com/paper/1905.06316/) (2019)
- [Analysis Methods in Neural Language Processing: A Survey](https://mlsanity.com/paper/1812.08951/) (2018)
- [Fast Differentiable Sorting and Ranking](https://mlsanity.com/paper/2002.08871/) (2020)
- [A Weighted Correlation Index for Rankings with Ties](https://mlsanity.com/paper/1404.3325/) (2014)
- [Probing Classifiers are Unreliable for Concept Removal and Detection](https://mlsanity.com/paper/2207.04153/) (2022)
- [Joint processing of linguistic properties in brains and language models](https://mlsanity.com/paper/2212.08094/) (2022)
- [Neural Language Models are not Born Equal to Fit Brain Data, but Training Helps](https://mlsanity.com/paper/2207.03380/) (2022)
- [Disentangling Semantics and Syntax in Sentence Embeddings with Pre-trained Language Models](https://mlsanity.com/paper/2104.05115/) (2021)
- [Interpreting Encoding and Decoding Models](https://mlsanity.com/paper/1812.00278/) (2018)
- [Probing for Constituency Structure in Neural Language Models](https://mlsanity.com/paper/2204.06201/) (2022)
- [Information-Restricted Neural Language Models Reveal Different Brain Regions' Sensitivity to Semantics, Syntax and Context](https://mlsanity.com/paper/2302.14389/) (2023)
- [Long-range and hierarchical language predictions in brains and algorithms](https://mlsanity.com/paper/2111.14232/) (2021)
- [Variable Importance in High-Dimensional Settings Requires Grouping](https://mlsanity.com/paper/2312.10858/) (2023)
- [Parametrization Cookbook: A set of Bijective Parametrizations for using Machine Learning methods in Statistical Inference](https://mlsanity.com/paper/2301.08297/) (2023)
- [Metric Learning for Phoneme Perception](https://mlsanity.com/paper/1809.07824/) (2018)

## Built on by (most cited first, 3 of 3)

- [What Makes Two Language Models Think Alike?](https://mlsanity.com/paper/2406.12620/) (2024)
- [No Such Thing as a General Learner: Language models and their dual optimization](https://mlsanity.com/paper/2408.09544/) (2024)
- [A Neural Model for Word Repetition](https://mlsanity.com/paper/2506.13450/) (2025)

## Abstract (arXiv)

Understanding how explicit theoretical features are encoded in opaque neural systems is a central challenge now common to neuroscience and AI. We introduce Metric Learning Encoding Models (MLEMs) to address this challenge most directly as a metric learning problem: we fit the distance in the space of theoretical features to match the distance in neural space. Our framework improves on univariate encoding and decoding methods by building on second-order isomorphism methods, such as Representational Similarity Analysis, and extends them by learning a metric that efficiently models feature as well as interactions between them. The effectiveness of MLEM is validated through two sets of simulations. First, MLEMs recover ground-truth importance features in synthetic datasets better than state-of-the-art methods, such as Feature Reweighted RSA (FR-RSA). Second, we deploy MLEMs on real language data, where they show stronger robustness to noise in calculating the importance of linguistic features (gender, tense, etc.). MLEMs are applicable to any domains where theoretical features can be identified, such as language, vision, audition, etc. We release optimized code applicable to measure feature importance in the representations of any artificial neural networks or empirical neural data at https://github.com/LouisJalouzot/MLEM.