# Interpreting Encoding and Decoding Models

arXiv: https://arxiv.org/abs/1812.00278 · HTML: https://mlsanity.com/paper/1812.00278/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Dec 2018.

## Built on by (most cited first, 21 of 21)

- [Neural Language Taskonomy: Which NLP Tasks are the most Predictive of fMRI Brain Activity?](https://mlsanity.com/paper/2205.01404/) (2022)
- [What Representational Similarity Measures Imply about Decodable Information](https://mlsanity.com/paper/2411.08197/) (2024)
- [Probing artificial neural networks: insights from neuroscience](https://mlsanity.com/paper/2104.08197/) (2021)
- [Representational dissimilarity metric spaces for stochastic neural networks](https://mlsanity.com/paper/2211.11665/) (2022)
- [Mapping Brains with Language Models: A Survey](https://mlsanity.com/paper/2306.05126/) (2023)
- [Multilevel Interpretability Of Artificial Neural Networks: Leveraging Framework And Methods From Neuroscience](https://mlsanity.com/paper/2408.12664/) (2024)
- [MSG-BART: Multi-granularity Scene Graph-Enhanced Encoder-Decoder Language Model for Video-grounded Dialogue Generation](https://mlsanity.com/paper/2311.12820/) (2023)
- [Metric Learning Encoding Models: A Multivariate Framework for Interpreting Neural Representations](https://mlsanity.com/paper/2402.11608/) (2024)
- [Modality-Agnostic fMRI Decoding of Vision and Language](https://mlsanity.com/paper/2403.11771/) (2024)
- [Can We Use Probing to Better Understand Fine-tuning and Knowledge Distillation of the BERT NLU?](https://mlsanity.com/paper/2301.11688/) (2023)
- [What Makes Two Language Models Think Alike?](https://mlsanity.com/paper/2406.12620/) (2024)
- [Exploring Deep Learning Models for EEG Neural Decoding](https://mlsanity.com/paper/2503.16567/) (2025)
- [Neuro2Semantic: A Transfer Learning Framework for Semantic Reconstruction of Continuous Language from Human Intracranial EEG](https://mlsanity.com/paper/2506.00381/) (2025)
- [Statistical Opportunities in Neuroimaging](https://mlsanity.com/paper/2602.12974/) (2026)
- [Rethinking Brain Decoding with CLIP: The Role of Adversarial Robustness](https://mlsanity.com/paper/2607.03165/) (2026)
- [Using Single-Trial Representational Similarity Analysis with EEG to track semantic similarity in emotional word processing](https://mlsanity.com/paper/2110.03529/) (2021)
- [STN: a new tensor network method to identify stimulus category from brain activity pattern](https://mlsanity.com/paper/2210.16993/) (2022)
- [Generalizability analysis of deep learning predictions of human brain responses to augmented and semantically novel visual stimuli](https://mlsanity.com/paper/2410.04497/) (2024)
- [The Prompting Brain: Neurocognitive Markers of Expertise in Guiding Large Language Models](https://mlsanity.com/paper/2508.14869/) (2025)
- [MEIcoder: Decoding Visual Stimuli from Neural Activity by Leveraging Most Exciting Inputs](https://mlsanity.com/paper/2510.20762/) (2025)
- [Logical subspace in LLMs](https://mlsanity.com/paper/2609.32907/) (2026)

## Abstract (arXiv)

Encoding and decoding models are widely used in systems, cognitive, and computational neuroscience to make sense of brain-activity data. However, the interpretation of their results requires care. Decoding models can help reveal whether particular information is present in a brain region in a format the decoder can exploit. Encoding models make comprehensive predictions about representational spaces. In the context of sensory systems, encoding models enable us to test and compare brain-computational models, and thus directly constrain computational theory. Encoding and decoding models typically include fitted linear-model components. Sometimes the weights of the fitted linear combinations are interpreted as reflecting, in an encoding model, the contribution of different sensory features to the representation or, in a decoding model, the contribution of different measured brain responses to a decoded feature. Such interpretations can be problematic when the predictor variables or their noise components are correlated and when priors (or penalties) are used to regularize the fit. Encoding and decoding models are evaluated in terms of their generalization performance. The correct interpretation depends on the level of generalization a model achieves (e.g. to new response measurements for the same stimuli, to new stimuli from the same population, or to stimuli from a different population). Significant decoding or encoding performance of a single model (at whatever level of generality) does not provide strong constraints for theory. Many models must be tested and inferentially compared for analyses to drive theoretical progress.