# Probing Brain Context-Sensitivity with Masked-Attention Generation

arXiv: https://arxiv.org/abs/2305.13863 · HTML: https://mlsanity.com/paper/2305.13863/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 3 of its 15 references that are in the map.
- First posted on arXiv 23 May 2023.

## Builds on (most cited first, 3 of 3)

- [Fine-grain atlases of functional modes for fMRI analysis](https://mlsanity.com/paper/2003.05405/) (2020)
- [Neural Language Models are not Born Equal to Fit Brain Data, but Training Helps](https://mlsanity.com/paper/2207.03380/) (2022)
- [Information-Restricted Neural Language Models Reveal Different Brain Regions' Sensitivity to Semantics, Syntax and Context](https://mlsanity.com/paper/2302.14389/) (2023)

## Abstract (arXiv)

Two fundamental questions in neurolinguistics concerns the brain regions that integrate information beyond the lexical level, and the size of their window of integration. To address these questions we introduce a new approach named masked-attention generation. It uses GPT-2 transformers to generate word embeddings that capture a fixed amount of contextual information. We then tested whether these embeddings could predict fMRI brain activity in humans listening to naturalistic text. The results showed that most of the cortex within the language network is sensitive to contextual information, and that the right hemisphere is more sensitive to longer contexts than the left. Masked-attention generation supports previous analyses of context-sensitivity in the brain, and complements them by quantifying the window size of context integration per voxel.