Phoneme- vs. Character-Level Targets and Selective State-Space Models for Intracortical Brain-to-Text
On the map: State-space sequence models (S4, Mamba) → Core SSM architectures for language
Abstract (arXiv)
State-of-the-art intracortical brain-to-text systems pair a neural-sequence phone decoder with an external language model. Two design axes remain underexplored: whether selective state-space models (Mamba) improve on recurrent decoders, and how the output target (phonetic vs. character) interacts with that choice. On the public Brain-to-Text '25 benchmark, we study a controlled 2x2 grid (GRU vs. hybrid Mamba decoder; phonetic vs. character targets) trained with a CTC objective under one reproducible protocol. The recurrent baseline remains strongest: the best phonetic GRU reaches 12.62% PER and 21.19% WER, while the best textual GRU after LM rescoring reaches 13.39% CER and 26.28% WER. The Mamba hybrid is competitive but does not surpass it. Ablations isolate architectural contributions, and error analysis shows representation-dependent failures: articulatory-like phoneme confusions vs. lexical and word-boundary errors.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Core SSM architectures for language, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 17 of its 50 references that are in the map. First posted on arXiv 29 Jul 2026.
Builds on
17 of 50 references.
Closest ancestors · 1
Same direction, newest first
Other references · 16
- 2017Attention Is All You Needcited by 60,361
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
- 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
- 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,380
- 2015Listen, Attend and Spellcited by 543
- 2022Decoding speech perception from non-invasive brain recordingscited by 130
- 2022Branchformer: Parallel MLP-Attention Architectures to Capture Local and Global Context for Speech Recognition and Understandingcited by 91
- 2021Efficient conformer: Progressive downsampling and grouped attention for automatic speech recognitioncited by 42
- 2024Speech Slytherin: Examining the Performance and Efficiency of Mamba for Speech Separation, Recognition, and Synthesiscited by 23
4 more
- 2024Exploring the Capability of Mamba in Speech Applicationscited by 20
- 2024Brain-to-Text Benchmark '24: Lessons Learnedcited by 8
- 2024Brain-to-Text Decoding with Context-Aware Neural Representations and Large Language Modelscited by 7
- 2026End-to-End Intracortical Speech Decoding from Neural Activitycited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Linguistic properties and model scale in brain encoding: from small to compressed language models
- 2024An Empirical Study of Mamba-based Language Models
- 2026Interpreting Brain Responses to Language with Sparse Features from Language Models
- 2026The Platonic brain bridge hypothesis: human brain networks as an architectural prior for multimodal large language models
- 2025Neural Correlates of Language Models Are Specific to Human Language
- 2026The Capacity of Thought: Benchmarking Llama 3.2 in Semantic fMRI Neural Language Decoding and Improving the Huth Encoding-Model Baseline
- 2025BrainWavLM: Fine-tuning Speech Representations with Brain Responses to Language
- 2026Reliable Neural-Codec Text-to-Speech by ASR Self-Verification and Distillation: Near-Zero Catastrophic Failures Across Models and Codecs