# Spatial encoding of BOLD fMRI time series for categorizing static images across visual datasets: A pilot study on human vision

arXiv: https://arxiv.org/abs/2309.03590 · HTML: https://mlsanity.com/paper/2309.03590/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 16 references that are in the map.
- First posted on arXiv 7 Sep 2023.

## Builds on (most cited first, 5 of 5)

- [Imaging Time-Series to Improve Classification and Imputation](https://mlsanity.com/paper/1506.00327/) (2015)
- [BOLD5000: A public fMRI dataset of 5000 images](https://mlsanity.com/paper/1809.01281/) (2018)
- [Implementation of Deep Neural Networks to Classify EEG Signals using Gramian Angular Summation Field for Epilepsy Diagnosis](https://mlsanity.com/paper/2003.04534/) (2020)
- [A Tour of Visualization Techniques for Computer Vision Datasets](https://mlsanity.com/paper/2204.08601/) (2022)
- [Understanding Cognitive Fatigue from fMRI Scans with Self-supervised Learning](https://mlsanity.com/paper/2106.15009/) (2021)

## Built on by (most cited first, 6 of 6)

- [Image complexity based fMRI-BOLD visual network categorization across visual datasets using topological descriptors and deep-hybrid learning](https://mlsanity.com/paper/2311.08417/) (2023)
- [Multi-scale fMRI time series analysis for understanding neurodegeneration in MCI](https://mlsanity.com/paper/2402.02811/) (2024)
- [Investigating the changes in BOLD responses during viewing of images with varied complexity: An fMRI time-series based analysis on human vision](https://mlsanity.com/paper/2309.15495/) (2023)
- [Towards understanding the nature of direct functional connectivity in visual brain network](https://mlsanity.com/paper/2403.11480/) (2024)
- [Non-linear Analysis Based ECG Classification of Cardiovascular Disorders](https://mlsanity.com/paper/2408.01542/) (2024)
- [Deep learning and whole-brain networks for biomarker discovery: modeling the dynamics of brain fluctuations in resting-state and cognitive tasks](https://mlsanity.com/paper/2412.19329/) (2024)

## Abstract (arXiv)

Functional MRI (fMRI) is widely used to examine brain functionality by detecting alteration in oxygenated blood flow that arises with brain activity. In this study, complexity specific image categorization across different visual datasets is performed using fMRI time series (TS) to understand differences in neuronal activities related to vision. Publicly available BOLD5000 dataset is used for this purpose, containing fMRI scans while viewing 5254 images of diverse categories, drawn from three standard computer vision datasets: COCO, ImageNet and SUN. To understand vision, it is important to study how brain functions while looking at different images. To achieve this, spatial encoding of fMRI BOLD TS has been performed that uses classical Gramian Angular Field (GAF) and Markov Transition Field (MTF) to obtain 2D BOLD TS, representing images of COCO, Imagenet and SUN. For classification, individual GAF and MTF features are fed into regular CNN. Subsequently, parallel CNN model is employed that uses combined 2D features for classifying images across COCO, Imagenet and SUN. The result of 2D CNN models is also compared with 1D LSTM and Bi-LSTM that utilizes raw fMRI BOLD signal for classification. It is seen that parallel CNN model outperforms other network models with an improvement of 7% for multi-class classification. Clinical relevance- The obtained result of this analysis establishes a baseline in studying how differently human brain functions while looking at images of diverse complexities.