# Transfer learning to decode brain states reflecting the relationship between cognitive tasks

arXiv: https://arxiv.org/abs/2206.03950 · HTML: https://mlsanity.com/paper/2206.03950/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 56 references that are in the map.
- First posted on arXiv 7 Jun 2022.

## Builds on (most cited first, 8 of 8)

- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [Taskonomy: Disentangling Task Transfer Learning](https://mlsanity.com/paper/1804.08328/) (2018)
- [Task2Vec: Task Embedding for Meta-Learning](https://mlsanity.com/paper/1902.03545/) (2019)
- [Representation Similarity Analysis for Efficient Task taxonomy & Transfer Learning](https://mlsanity.com/paper/1904.11740/) (2019)
- [Partial Transfer Learning with Selective Adversarial Networks](https://mlsanity.com/paper/1707.07901/) (2017)
- [Neural system identification for large populations separating "what" and "where"](https://mlsanity.com/paper/1711.02653/) (2017)
- [Decoding and mapping task states of the human brain via deep learning](https://mlsanity.com/paper/1801.09858/) (2018)
- [Deep Auto-encoder with Neural Response](https://mlsanity.com/paper/2111.15309/) (2021)

## Built on by (most cited first, 5 of 5)

- [Evaluating the structure of cognitive tasks with transfer learning](https://mlsanity.com/paper/2308.02408/) (2023)
- [Uncovering cognitive taskonomy through transfer learning in masked autoencoder-based fMRI reconstruction](https://mlsanity.com/paper/2407.00033/) (2024)
- [FlexiBrain: Resolution-Agnostic Voxel-Level Encoding for Native fMRI](https://mlsanity.com/paper/2606.11500/) (2026)
- [A Hybrid Brain-Computer Interface Using Motor Imagery and SSVEP Based on Convolutional Neural Network](https://mlsanity.com/paper/2212.05289/) (2022)
- [Large-scale Foundation Models and Generative AI for BigData Neuroscience](https://mlsanity.com/paper/2310.18377/) (2023)

## Abstract (arXiv)

Transfer learning improves the performance of the target task by leveraging the data of a specific source task: the closer the relationship between the source and the target tasks, the greater the performance improvement by transfer learning. In neuroscience, the relationship between cognitive tasks is usually represented by similarity of activated brain regions or neural representation. However, no study has linked transfer learning and neuroscience to reveal the relationship between cognitive tasks. In this study, we propose a transfer learning framework to reflect the relationship between cognitive tasks, and compare the task relations reflected by transfer learning and by the overlaps of brain regions (e.g., neurosynth). Our results of transfer learning create cognitive taskonomy to reflect the relationship between cognitive tasks which is well in line with the task relations derived from neurosynth. Transfer learning performs better in task decoding with fMRI data if the source and target cognitive tasks activate similar brain regions. Our study uncovers the relationship of multiple cognitive tasks and provides guidance for source task selection in transfer learning for neural decoding based on small-sample data.