# A Penny for Your Thoughts: Decoding Speech from Inexpensive Brain Signals

arXiv: https://arxiv.org/abs/2511.04691 · HTML: https://mlsanity.com/paper/2511.04691/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 9 references that are in the map.
- First posted on arXiv 28 Oct 2025.

## Builds on (most cited first, 3 of 3)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Decoding speech perception from non-invasive brain recordings](https://mlsanity.com/paper/2208.12266/) (2022)
- [Converting Your Thoughts to Texts: Enabling Brain Typing via Deep Feature Learning of EEG Signals](https://mlsanity.com/paper/1709.08820/) (2017)

## Built on by (most cited first, 2 of 2)

- [MultiDiffNet: A Multi-Objective Diffusion Framework for Generalizable Brain Decoding](https://mlsanity.com/paper/2511.18294/) (2025)
- [Aligning Brain Signals with Multimodal Speech and Vision Embeddings](https://mlsanity.com/paper/2511.00065/) (2025)

## Abstract (arXiv)

We explore whether neural networks can decode brain activity into speech by mapping EEG recordings to audio representations. Using EEG data recorded as subjects listened to natural speech, we train a model with a contrastive CLIP loss to align EEG-derived embeddings with embeddings from a pre-trained transformer-based speech model. Building on the state-of-the-art EEG decoder from Meta, we introduce three architectural modifications: (i) subject-specific attention layers (+0.15% WER improvement), (ii) personalized spatial attention (+0.45%), and (iii) a dual-path RNN with attention (-1.87%). Two of the three modifications improved performance, highlighting the promise of personalized architectures for brain-to-speech decoding and applications in brain-computer interfaces.