# Leveraging sinusoidal representation networks to predict fMRI signals from EEG

arXiv: https://arxiv.org/abs/2311.04234 · HTML: https://mlsanity.com/paper/2311.04234/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 20 references that are in the map.
- First posted on arXiv 6 Nov 2023.

## Builds on (most cited first, 6 of 6)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Time2Vec: Learning a Vector Representation of Time](https://mlsanity.com/paper/1907.05321/) (2019)
- [DC-UNet: Rethinking the U-Net Architecture with Dual Channel Efficient CNN for Medical Images Segmentation](https://mlsanity.com/paper/2006.00414/) (2020)
- [Min-Max Similarity: A Contrastive Semi-Supervised Deep Learning Network for Surgical Tools Segmentation](https://mlsanity.com/paper/2203.15177/) (2022)
- [fMRI from EEG is only Deep Learning away: the use of interpretable DL to unravel EEG-fMRI relationships](https://mlsanity.com/paper/2211.02024/) (2022)

## Built on by (most cited first, 6 of 6)

- [SineKAN: Kolmogorov-Arnold Networks Using Sinusoidal Activation Functions](https://mlsanity.com/paper/2407.04149/) (2024)
- [NeuroBOLT: Resting-state EEG-to-fMRI Synthesis with Multi-dimensional Feature Mapping](https://mlsanity.com/paper/2410.05341/) (2024)
- [Spec2VolCAMU-Net: A Spectrogram-to-Volume Model for EEG-to-fMRI Reconstruction based on Multi-directional Time-Frequency Convolutional Attention Encoder and Vision-Mamba U-Net](https://mlsanity.com/paper/2505.09521/) (2025)
- [DaRePlane: Direction-aware Representations for Dynamic Scene Reconstruction](https://mlsanity.com/paper/2410.14169/) (2024)
- [EEG-to-fMRI synthesis of task-evoked and spontaneous brain activity: addressing issues of statistical significance and generalizability](https://mlsanity.com/paper/2504.10752/) (2025)
- [MEL: Coordinate-Preserving EEG Tokenization for fMRI Translation](https://mlsanity.com/paper/2608.29304/) (2026)

## Abstract (arXiv)

In modern neuroscience, functional magnetic resonance imaging (fMRI) has been a crucial and irreplaceable tool that provides a non-invasive window into the dynamics of whole-brain activity. Nevertheless, fMRI is limited by hemodynamic blurring as well as high cost, immobility, and incompatibility with metal implants. Electroencephalography (EEG) is complementary to fMRI and can directly record the cortical electrical activity at high temporal resolution, but has more limited spatial resolution and is unable to recover information about deep subcortical brain structures. The ability to obtain fMRI information from EEG would enable cost-effective, imaging across a wider set of brain regions. Further, beyond augmenting the capabilities of EEG, cross-modality models would facilitate the interpretation of fMRI signals. However, as both EEG and fMRI are high-dimensional and prone to artifacts, it is currently challenging to model fMRI from EEG. To address this challenge, we propose a novel architecture that can predict fMRI signals directly from multi-channel EEG without explicit feature engineering. Our model achieves this by implementing a Sinusoidal Representation Network (SIREN) to learn frequency information in brain dynamics from EEG, which serves as the input to a subsequent encoder-decoder to effectively reconstruct the fMRI signal from a specific brain region. We evaluate our model using a simultaneous EEG-fMRI dataset with 8 subjects and investigate its potential for predicting subcortical fMRI signals. The present results reveal that our model outperforms a recent state-of-the-art model, and indicates the potential of leveraging periodic activation functions in deep neural networks to model functional neuroimaging data.