The Capacity of Thought: Benchmarking Llama 3.2 in Semantic fMRI Neural Language Decoding and Improving the Huth Encoding-Model Baseline
On the map: LLM reasoning → Reasoning benchmarks and evaluation
Abstract (arXiv)
Decoding continuous language from fMRI signals remains a core challenge in non-invasive brain-computer interface research. We present two complementary investigations. First, we improve the Huth et al. ridge regression encoding pipeline through expanded voxel selection (10K->15K), substitution of GPT-2 medium for GPT-1 as the beam-search proposal model, and GPU-accelerated bootstrap training, achieving mean METEOR = 0.149 and BLEU-1 = 0.200 across three held-out narratives for subject UTS03 -- an 11% relative METEOR gain over our replication baseline. Second, we introduce fMRIFlamingo, which maps BOLD activity to a frozen Llama-3.2-1B with trainable gated cross-attention layers via a learned brain tokenizer and a Perceiver Resampler. Despite achieving 42.86% Top-1 accuracy on a 1-in-100 ranking task, well above chance, a blind control ablation with zeroed fMRI inputs yields near-identical scores, revealing that apparent decoding success is driven primarily by the frozen language prior rather than by neural input. These results demonstrate that high-capacity language models do not inherently improve fMRI decoding and can actively obscure failures without rigorous blind-control evaluation.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 3 of its 8 references that are in the map. First posted on arXiv 13 Jul 2026.
Builds on
3 of 8 references.
Closest ancestors · 1
Same direction, newest first
Other references · 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025fMRI-LM: Towards a Universal Foundation Model for Language-Aligned fMRI Understanding
- 2024BrainChat: Decoding Semantic Information from fMRI using Vision-language Pretrained Models
- 2022Semantic Brain Decoding: from fMRI to conceptually similar image reconstruction of visual stimuli
- 2026Sparse Autoencoders Map Brain-LLM Alignment onto Cortical Semantic Topography
- 2025Decoding the Multimodal Mind: Generalizable Brain-to-Text Translation via Multimodal Alignment and Adaptive Routing
- 2024Aligned with LLM: a new multi-modal training paradigm for encoding fMRI activity in visual cortex
- 2026MindAlign: Decoding Inner Speech from fMRI Signals via Multimodal Embedding Alignment under Limited Data
- 2024A multimodal LLM for the non-invasive decoding of spoken text from brain recordings