# The Semantic Bottleneck: Leveraging Semantic Representations for Non-Invasive Speech Decoding

arXiv: https://arxiv.org/abs/2609.10296 · HTML: https://mlsanity.com/paper/2609.10296/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 32 references that are in the map.
- First posted on arXiv 9 Sep 2026.

## Builds on (most cited first, 18 of 18)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning](https://mlsanity.com/paper/2105.04906/) (2021)
- [MTEB: Massive Text Embedding Benchmark](https://mlsanity.com/paper/2210.07316/) (2022)
- [Decoding speech perception from non-invasive brain recordings](https://mlsanity.com/paper/2208.12266/) (2022)
- [SONAR: Sentence-Level Multimodal and Language-Agnostic Representations](https://mlsanity.com/paper/2308.11466/) (2023)
- [Manifold learning: what, how, and why](https://mlsanity.com/paper/2311.03757/) (2023)
- [Language Model Inversion](https://mlsanity.com/paper/2311.13647/) (2023)
- [Harnessing the Universal Geometry of Embeddings](https://mlsanity.com/paper/2505.12540/) (2025)
- [Are EEG-to-Text Models Working?](https://mlsanity.com/paper/2405.06459/) (2024)
- [LibriBrain: Over 50 Hours of Within-Subject MEG to Improve Speech Decoding Methods at Scale](https://mlsanity.com/paper/2506.02098/) (2025)
- [MAD: Multi-Alignment MEG-to-Text Decoding](https://mlsanity.com/paper/2406.01512/) (2024)
- [NeuSpeech: Decode Neural signal as Speech](https://mlsanity.com/paper/2403.01748/) (2024)
- [Brain-to-Text Benchmark '24: Lessons Learned](https://mlsanity.com/paper/2412.17227/) (2024)
- [NeuGPT: Unified multi-modal Neural GPT](https://mlsanity.com/paper/2410.20916/) (2024)
- [Semantic reconstruction of continuous language from MEG signals](https://mlsanity.com/paper/2309.07701/) (2023)
- [BrainECHO: Semantic Brain Signal Decoding through Vector-Quantized Spectrogram Reconstruction for Whisper-Enhanced Text Generation](https://mlsanity.com/paper/2410.14971/) (2024)

## Built on by (most cited first, 2 of 2)

- [Brain-to-Language Decoding: Tasks, Signals, Methods, Evaluation, Practical Use and Beyond](https://mlsanity.com/paper/2609.27650/) (2026)
- [Removing Timing Shortcuts Improves Non-Invasive Brain-to-Text](https://mlsanity.com/paper/2609.40359/) (2026)

## Abstract (arXiv)

Non-invasive speech decoding remains constrained by the low signal-to-noise ratio of neural recordings, which makes fine-grained reconstruction of phonemes or individual words difficult. Motivated by neuroscientific evidence that high-level semantic representations are distributed across cortical regions and evolve over slower temporal scales, we hypothesize that semantic content may provide a more suitable target for non-invasive decoding than low-level acoustic or lexical features. We introduce Brain2Semantics2Text, a method that reconstructs text through an intermediate semantic embedding space. Our model maps sentence-level MEG responses into a semantic manifold and then inverts the predicted embeddings into natural language. This semantic bottleneck enables recovery of high-level meaning without word-level alignment. We describe the core principles of the approach, its implementation, and the strategies used to mitigate the challenges of learning a reliable neural-to-semantic mapping. Finally, we compare against prior non-invasive Brain2Text methods and show improved sentence-level results.