# Fine-grained Analysis of Brain-LLM Alignment through Input Attribution

arXiv: https://arxiv.org/abs/2510.12355 · HTML: https://mlsanity.com/paper/2510.12355/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 23 of its 41 references that are in the map.
- First posted on arXiv 14 Oct 2025.

## Placed in

- LLM alignment / RLHF / preference optimisation / [LLM preference optimization methods](https://mlsanity.com/alignment/llm-preference-optimization-methods/)

## Builds on (most cited first, 23 of 23)

- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Captum: A unified and generic model interpretability library for PyTorch](https://mlsanity.com/paper/2009.07896/) (2020)
- [Not Just a Black Box: Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1605.01713/) (2016)
- [Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain)](https://mlsanity.com/paper/1905.11833/) (2019)
- [On the Emergence of Position Bias in Transformers](https://mlsanity.com/paper/2502.01951/) (2025)
- [Instructed to Bias: Instruction-Tuned Language Models Exhibit Emergent Cognitive Bias](https://mlsanity.com/paper/2308.00225/) (2023)
- [Joint processing of linguistic properties in brains and language models](https://mlsanity.com/paper/2212.08094/) (2022)
- [Contextual Feature Extraction Hierarchies Converge in Large Language Models and the Brain](https://mlsanity.com/paper/2401.17671/) (2024)
- [From Language to Cognition: How LLMs Outgrow the Human Language Network](https://mlsanity.com/paper/2503.01830/) (2025)
- [Training language models to summarize narratives improves brain alignment](https://mlsanity.com/paper/2212.10898/) (2022)
- [Improving Semantic Understanding in Speech Language Models via Brain-tuning](https://mlsanity.com/paper/2410.09230/) (2024)
- [Speech language models lack important brain-relevant semantics](https://mlsanity.com/paper/2311.04664/) (2023)
- [Understanding and Mitigating Bottlenecks of State Space Models through the Lens of Recency and Over-smoothing](https://mlsanity.com/paper/2501.00658/) (2024)
- [Language models and brains align due to more than next-word prediction and word-level information](https://mlsanity.com/paper/2212.00596/) (2022)
- [Token-wise Decomposition of Autoregressive Language Model Hidden States for Analyzing Model Predictions](https://mlsanity.com/paper/2305.10614/) (2023)
- [Exploiting Primacy Effect To Improve Large Language Models](https://mlsanity.com/paper/2507.13949/) (2025)
- [Emergence of the Primacy Effect in Structured State-Space Models](https://mlsanity.com/paper/2502.13729/) (2025)
- [Explanations of Large Language Models Explain Language Representations in the Brain](https://mlsanity.com/paper/2502.14671/) (2025)

## Built on by (most cited first, 3 of 3)

- [Linguistics and Human Brain: A Perspective of Computational Neuroscience](https://mlsanity.com/paper/2602.08275/) (2026)
- [Layer-wise Positional Bias in Short-Context Language Modeling](https://mlsanity.com/paper/2601.04098/) (2026)
- [Training-Driven Representational Geometry Modularization Predicts Brain Alignment in Language Models](https://mlsanity.com/paper/2602.07539/) (2026)

## Abstract (arXiv)

Understanding the alignment between large language models (LLMs) and human brain activity can reveal computational principles underlying language processing. We introduce a fine-grained input attribution method to identify the specific words most important for brain-LLM alignment, and leverage it to study a contentious research question about brain-LLM alignment: the relationship between brain alignment (BA) and next-word prediction (NWP). Our findings reveal that BA and NWP rely on largely distinct word subsets: NWP exhibits recency and primacy biases with a focus on syntax, while BA prioritizes semantic and discourse-level information with a more targeted recency effect. This work advances our understanding of how LLMs relate to human language processing and highlights differences in feature reliance between BA and NWP. Beyond this study, our attribution method can be broadly applied to explore the cognitive relevance of model predictions in diverse language processing tasks.