# Group-level Brain Decoding with Deep Learning

arXiv: https://arxiv.org/abs/2205.14102 · HTML: https://mlsanity.com/paper/2205.14102/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 69 references that are in the map.
- First posted on arXiv 27 May 2022.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [Personalizing Dialogue Agents: I have a dog, do you have pets too?](https://mlsanity.com/paper/1801.07243/) (2018)
- [Image Transformer](https://mlsanity.com/paper/1802.05751/) (2018)
- [EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces](https://mlsanity.com/paper/1611.08024/) (2016)
- [Interpretable machine learning: definitions, methods, and applications](https://mlsanity.com/paper/1901.04592/) (2019)
- [A Persona-Based Neural Conversation Model](https://mlsanity.com/paper/1603.06155/) (2016)
- [BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data](https://mlsanity.com/paper/2101.12037/) (2021)
- [MEG Decoding Across Subjects](https://mlsanity.com/paper/1404.4175/) (2014)
- [Adaptive neural network classifier for decoding MEG signals](https://mlsanity.com/paper/1805.10981/) (2018)
- [U-vectors: Generating clusterable speaker embedding from unlabeled data](https://mlsanity.com/paper/2102.03868/) (2021)
- [DNN-based Speaker Embedding Using Subjective Inter-speaker Similarity for Multi-speaker Modeling in Speech Synthesis](https://mlsanity.com/paper/1907.08294/) (2019)
- [MDCNN-SID: Multi-scale Dilated Convolution Network for Singer Identification](https://mlsanity.com/paper/2004.04371/) (2020)

## Built on by (most cited first, 7 of 7)

- [The Brain's Bitter Lesson: Scaling Speech Decoding With Self-Supervised Learning](https://mlsanity.com/paper/2406.04328/) (2024)
- [LibriBrain: Over 50 Hours of Within-Subject MEG to Improve Speech Decoding Methods at Scale](https://mlsanity.com/paper/2506.02098/) (2025)
- [MEG-GPT: A transformer-based foundation model for magnetoencephalography data](https://mlsanity.com/paper/2510.18080/) (2025)
- [GPT2MEG: Quantizing MEG for Autoregressive Generation](https://mlsanity.com/paper/2404.09256/) (2024)
- [Elementary, My Dear Watson: Non-Invasive Neural Keyword Spotting in the LibriBrain Dataset](https://mlsanity.com/paper/2510.21038/) (2025)
- [Non-invasive Neural Decoding in Source Reconstructed Brain Space](https://mlsanity.com/paper/2410.19838/) (2024)
- [Convolutional Monge Mapping Normalization for learning on sleep data](https://mlsanity.com/paper/2305.18831/) (2023)

## Abstract (arXiv)

Decoding brain imaging data are gaining popularity, with applications in brain-computer interfaces and the study of neural representations. Decoding is typicallysubject-specific and does not generalise well over subjects, due to high amounts ofbetween subject variability. Techniques that overcome this will not only providericher neuroscientific insights but also make it possible for group-level models to out-perform subject-specific models. Here, we propose a method that uses subjectembedding, analogous to word embedding in natural language processing, to learnand exploit the structure in between-subject variability as part of a decoding model,our adaptation of the WaveNet architecture for classification. We apply this to mag-netoencephalography data, where 15 subjects viewed 118 different images, with30 examples per image; to classify images using the entire 1 s window followingimage presentation. We show that the combination of deep learning and subjectembedding is crucial to closing the performance gap between subject- and group-level decoding models. Importantly, group models outperform subject models onlow-accuracy subjects (although slightly impair high-accuracy subjects) and can behelpful for initialising subject models. While we have not generally found group-levelmodels to perform better than subject-level models, the performance of groupmodelling is expected to be even higher with bigger datasets. In order to providephysiological interpretation at the group level, we make use of permutation featureimportance. This provides insights into the spatiotemporal and spectral informationencoded in the models. All code is available on GitHub (https://github.com/ricsinaruto/MEG-group-decode).