# From Neurons to Conversation: Speech Brain-Computer Interfaces

arXiv: https://arxiv.org/abs/2609.36736 · HTML: https://mlsanity.com/paper/2609.36736/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 162 references that are in the map.
- First posted on arXiv 29 Sep 2026.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture](https://mlsanity.com/paper/2301.08243/) (2023)
- [EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces](https://mlsanity.com/paper/1611.08024/) (2016)
- [Listen, Attend and Spell](https://mlsanity.com/paper/1508.01211/) (2015)
- [Decoding speech perception from non-invasive brain recordings](https://mlsanity.com/paper/2208.12266/) (2022)
- [Machine learning for neural decoding](https://mlsanity.com/paper/1708.00909/) (2017)
- [Making brain-machine interfaces robust to future neural variability](https://mlsanity.com/paper/1610.05872/) (2016)
- [Brain-to-Text Decoding: A Non-invasive Approach via Typing](https://mlsanity.com/paper/2502.17480/) (2025)
- [The Brain's Bitter Lesson: Scaling Speech Decoding With Self-Supervised Learning](https://mlsanity.com/paper/2406.04328/) (2024)
- [LibriBrain: Over 50 Hours of Within-Subject MEG to Improve Speech Decoding Methods at Scale](https://mlsanity.com/paper/2506.02098/) (2025)
- [Evidence from fMRI Supports a Two-Phase Abstraction Process in Language Models](https://mlsanity.com/paper/2409.05771/) (2024)
- [From Thought to Action: How a Hierarchy of Neural Dynamics Supports Language Production](https://mlsanity.com/paper/2502.07429/) (2025)
- [Time-Masked Transformers with Lightweight Test-Time Adaptation for Neural Speech Decoding](https://mlsanity.com/paper/2507.02800/) (2025)
- [A cross-species neural foundation model for end-to-end speech decoding](https://mlsanity.com/paper/2511.21740/) (2025)
- [Emergence of Language in the Developing Brain](https://mlsanity.com/paper/2512.05718/) (2025)
- [A Pre-trained Framework for Multilingual Brain Decoding Using Non-invasive Recordings](https://mlsanity.com/paper/2506.03214/) (2025)
- [Decoding the decoder: Contextual sequence-to-sequence modeling for intracortical speech decoding](https://mlsanity.com/paper/2603.20246/) (2026)
- [Temporal structure of the language hierarchy within small cortical patches](https://mlsanity.com/paper/2604.03021/) (2026)

## Abstract (arXiv)

Speech brain-computer interfaces (BCIs) aim to restore communication by transforming neural activity related to speech, language, or communicative intent into external outputs such as text, synthesized voice, or avatar control. Recent advances in intracortical and electrocorticographic recording, deep sequence models, and language-model-assisted decoding have enabled rapid progress, including high-performance attempted-speech decoding and increasingly naturalistic speech synthesis. Yet these achievements also reveal that speech BCIs are not simply neural-to-text decoders. They are adaptive clinical systems in which neural representations, recording hardware, decoding architectures, language priors, feedback, and user learning interact over time. Here, we synthesize speech BCI research from a system-level perspective. We first examine the neural substrates of speech and language, emphasizing their hierarchical, distributed, temporally structured, and non-stationary organization. We then examine recording and decoding choices, closed-loop adaptation, evaluation, clinical translation, and ethics. Across these domains, we highlight recurring trade-offs between signal resolution and invasiveness, low-level motor and high-level semantic targets, decoder accuracy and user agency, and language-model fluency and faithful neural evidence. We argue the next generation of speech BCIs should be evaluated not only by offline accuracy, but also by robustness across sessions, calibration burden, latency, uncertainty, usability, and safeguards against unintended decoding. By reframing speech BCIs as adaptive, user-centred systems, we outline the interdisciplinary priorities spanning speech neuroscience, neural engineering, machine learning, clinical practice, and neuroethics needed to move from proof-of-concept decoding toward reliable, expressive, and controllable communication neuroprostheses.