# On the benefits of self-taught learning for brain decoding

arXiv: https://arxiv.org/abs/2209.10099 · HTML: https://mlsanity.com/paper/2209.10099/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 66 references that are in the map.
- First posted on arXiv 19 Sep 2022.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [Revisiting Unreasonable Effectiveness of Data in Deep Learning Era](https://mlsanity.com/paper/1707.02968/) (2017)
- [What is being transferred in transfer learning?](https://mlsanity.com/paper/2008.11687/) (2020)
- [Machine Learning for Neuroimaging with Scikit-Learn](https://mlsanity.com/paper/1412.3919/) (2014)
- [Leakage and the Reproducibility Crisis in ML-based Science](https://mlsanity.com/paper/2207.07048/) (2022)
- [How much data is needed to train a medical image deep learning system to achieve necessary high accuracy?](https://mlsanity.com/paper/1511.06348/) (2015)
- [Self-Supervised Learning of Brain Dynamics from Broad Neuroimaging Data](https://mlsanity.com/paper/2206.11417/) (2022)
- [Assessing and tuning brain decoders: cross-validation, caveats, and guidelines](https://mlsanity.com/paper/1606.05201/) (2016)
- [FMRI data augmentation via synthesis](https://mlsanity.com/paper/1907.06134/) (2019)
- [Decoding and mapping task states of the human brain via deep learning](https://mlsanity.com/paper/1801.09858/) (2018)
- [Deep learning of fMRI big data: a novel approach to subject-transfer decoding](https://mlsanity.com/paper/1502.00093/) (2015)
- [Extracting representations of cognition across neuroimaging studies improves brain decoding](https://mlsanity.com/paper/1809.06035/) (2018)
- [Few-shot Decoding of Brain Activation Maps](https://mlsanity.com/paper/2010.12500/) (2020)
- [Challenges for cognitive decoding using deep learning methods](https://mlsanity.com/paper/2108.06896/) (2021)
- [Evaluating deep transfer learning for whole-brain cognitive decoding](https://mlsanity.com/paper/2111.01562/) (2021)
- [Pipeline-Invariant Representation Learning for Neuroimaging](https://mlsanity.com/paper/2208.12909/) (2022)

## Built on by (most cited first, 1 of 1)

- [An Ensemble-Based Self-Taught Learning Approach for Parking Space Classification Under Limited Data](https://mlsanity.com/paper/2609.03258/) (2026)

## Abstract (arXiv)

Context. We study the benefits of using a large public neuroimaging database composed of fMRI statistic maps, in a self-taught learning framework, for improving brain decoding on new tasks. First, we leverage the NeuroVault database to train, on a selection of relevant statistic maps, a convolutional autoencoder to reconstruct these maps. Then, we use this trained encoder to initialize a supervised convolutional neural network to classify tasks or cognitive processes of unseen statistic maps from large collections of the NeuroVault database. Results. We show that such a self-taught learning process always improves the performance of the classifiers but the magnitude of the benefits strongly depends on the number of samples available both for pre-training and finetuning the models and on the complexity of the targeted downstream task. Conclusion. The pre-trained model improves the classification performance and displays more generalizable features, less sensitive to individual differences.