# ConvConcatNet: a deep convolutional neural network to reconstruct mel spectrogram from the EEG

arXiv: https://arxiv.org/abs/2401.04965 · HTML: https://mlsanity.com/paper/2401.04965/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 5 references that are in the map.
- First posted on arXiv 10 Jan 2024.

## Builds on (most cited first, 2 of 2)

- [Decoding speech perception from non-invasive brain recordings](https://mlsanity.com/paper/2208.12266/) (2022)
- [A DenseNet-based method for decoding auditory spatial attention with EEG](https://mlsanity.com/paper/2309.07690/) (2023)

## Built on by (most cited first, 12 of 12)

- [Neural Codecs as Biosignal Tokenizers](https://mlsanity.com/paper/2510.09095/) (2025)
- [Toward Fully-End-to-End Listened Speech Decoding from EEG Signals](https://mlsanity.com/paper/2406.08644/) (2024)
- [Enhancing Listened Speech Decoding from EEG via Parallel Phoneme Sequence Prediction](https://mlsanity.com/paper/2501.04844/) (2025)
- [MEGConformer: Conformer-Based MEG Decoder for Robust Speech and Phoneme Classification](https://mlsanity.com/paper/2512.01443/) (2025)
- [An Approach to Simultaneous Acquisition of Real-Time MRI Video, EEG, and Surface EMG for Articulatory, Brain, and Muscle Activity During Speech Production](https://mlsanity.com/paper/2603.04840/) (2026)
- [Cross-Subject Generalization in Decoding Perceived Speech from Non-Invasive Brain Recordings](https://mlsanity.com/paper/2608.22420/) (2026)
- [DPI-TTS: Directional Patch Interaction for Fast-Converging and Style Temporal Modeling in Text-to-Speech](https://mlsanity.com/paper/2409.11835/) (2024)
- [SHINE: Sequential Hierarchical Integration Network for EEG and MEG](https://mlsanity.com/paper/2602.23960/) (2026)
- [MEG-to-MEG Transfer Learning and Cross-Task Speech/Silence Detection with Limited Data](https://mlsanity.com/paper/2602.18253/) (2026)
- [MEBM-Speech: Multi-scale Enhanced BrainMagic for Robust MEG Speech Detection](https://mlsanity.com/paper/2603.02255/) (2026)
- [Beyond Parallel Tracking: Interactive Multi-Feature Fusion Drives Semantic Reconstruction from Non-invasive Brain Recordings](https://mlsanity.com/paper/2607.12071/) (2026)
- [Joint Text-Audio Alignment for EEG-to-Text Decoding in Chinese Speech Production and Perception](https://mlsanity.com/paper/2607.25626/) (2026)

## Abstract (arXiv)

To investigate the processing of speech in the brain, simple linear models are commonly used to establish a relationship between brain signals and speech features. However, these linear models are ill-equipped to model a highly dynamic and complex non-linear system like the brain. Although non-linear methods with neural networks have been developed recently, reconstructing unseen stimuli from unseen subjects' EEG is still a highly challenging task. This work presents a novel method, ConvConcatNet, to reconstruct mel-specgrams from EEG, in which the deep convolution neural network and extensive concatenation operation were combined. With our ConvConcatNet model, the Pearson correlation between the reconstructed and the target mel-spectrogram can achieve 0.0420, which was ranked as No.1 in the Task 2 of the Auditory EEG Challenge. The codes and models to implement our work will be available on Github: https://github.com/xuxiran/ConvConcatNet