# NeuroBOLT: Resting-state EEG-to-fMRI Synthesis with Multi-dimensional Feature Mapping

arXiv: https://arxiv.org/abs/2410.05341 · HTML: https://mlsanity.com/paper/2410.05341/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 72 references that are in the map.
- First posted on arXiv 7 Oct 2024.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020)
- [Linformer: Self-Attention with Linear Complexity](https://mlsanity.com/paper/2006.04768/) (2020)
- [TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis](https://mlsanity.com/paper/2210.02186/) (2022)
- [Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCI](https://mlsanity.com/paper/2405.18765/) (2024)
- [Seeing Beyond the Brain: Conditional Diffusion Model with Sparse Masked Modeling for Vision Decoding](https://mlsanity.com/paper/2211.06956/) (2022)
- [Transformer-based Spatial-Temporal Feature Learning for EEG Decoding](https://mlsanity.com/paper/2106.11170/) (2021)
- [Cinematic Mindscapes: High-quality Video Reconstruction from Brain Activity](https://mlsanity.com/paper/2305.11675/) (2023)
- [Transformer Convolutional Neural Networks for Automated Artifact Detection in Scalp EEG](https://mlsanity.com/paper/2208.02405/) (2022)
- [CaraNet: Context Axial Reverse Attention Network for Segmentation of Small Medical Objects](https://mlsanity.com/paper/2301.13366/) (2023)
- [Self-Supervised Learning of Brain Dynamics from Broad Neuroimaging Data](https://mlsanity.com/paper/2206.11417/) (2022)
- [DC-UNet: Rethinking the U-Net Architecture with Dual Channel Efficient CNN for Medical Images Segmentation](https://mlsanity.com/paper/2006.00414/) (2020)
- [Min-Max Similarity: A Contrastive Semi-Supervised Deep Learning Network for Surgical Tools Segmentation](https://mlsanity.com/paper/2203.15177/) (2022)
- [Fine-grain atlases of functional modes for fMRI analysis](https://mlsanity.com/paper/2003.05405/) (2020)
- [DaReNeRF: Direction-aware Representation for Dynamic Scenes](https://mlsanity.com/paper/2403.02265/) (2024)
- [CFPNet-M: A Light-Weight Encoder-Decoder Based Network for Multimodal Biomedical Image Real-Time Segmentation](https://mlsanity.com/paper/2105.04075/) (2021)
- [Zero-Shot Surgical Tool Segmentation in Monocular Video Using Segment Anything Model 2](https://mlsanity.com/paper/2408.01648/) (2024)
- [WS-SfMLearner: Self-supervised Monocular Depth and Ego-motion Estimation on Surgical Videos with Unknown Camera Parameters](https://mlsanity.com/paper/2308.11776/) (2023)
- [EEG to fMRI Synthesis: Is Deep Learning a candidate?](https://mlsanity.com/paper/2009.14133/) (2020)
- [SAMSNeRF: Segment Anything Model (SAM) Guides Dynamic Surgical Scene Reconstruction by Neural Radiance Field (NeRF)](https://mlsanity.com/paper/2308.11774/) (2023)
- [ATD: Augmenting CP Tensor Decomposition by Self Supervision](https://mlsanity.com/paper/2106.07900/) (2021)
- [EEG to fMRI Synthesis Benefits from Attentional Graphs of Electrode Relationships](https://mlsanity.com/paper/2203.03481/) (2022)
- [Self-Supervised Surgical Instrument 3D Reconstruction from a Single Camera Image](https://mlsanity.com/paper/2211.14467/) (2022)
- [Leveraging sinusoidal representation networks to predict fMRI signals from EEG](https://mlsanity.com/paper/2311.04234/) (2023)
- [fMRI from EEG is only Deep Learning away: the use of interpretable DL to unravel EEG-fMRI relationships](https://mlsanity.com/paper/2211.02024/) (2022)
- [Bayesian fusion and multimodal DCM for EEG and fMRI](https://mlsanity.com/paper/1906.07354/) (2019)

## Built on by (most cited first, 11 of 11)

- [CineBrain: A Large-Scale Multi-Modal Brain Dataset During Naturalistic Audiovisual Narrative Processing](https://mlsanity.com/paper/2503.06940/) (2025)
- [Spec2VolCAMU-Net: A Spectrogram-to-Volume Model for EEG-to-fMRI Reconstruction based on Multi-directional Time-Frequency Convolutional Attention Encoder and Vision-Mamba U-Net](https://mlsanity.com/paper/2505.09521/) (2025)
- [Bridging Brain with Foundation Models through Self-Supervised Learning](https://mlsanity.com/paper/2506.16009/) (2025)
- [S2M-Former: Spiking Symmetric Mixing Branchformer for Brain Auditory Attention Detection](https://mlsanity.com/paper/2508.05164/) (2025)
- [Bridging Brain and Semantics: A Hierarchical Framework for Semantically Enhanced fMRI-to-Video Reconstruction](https://mlsanity.com/paper/2605.14569/) (2026)
- [DaRePlane: Direction-aware Representations for Dynamic Scene Reconstruction](https://mlsanity.com/paper/2410.14169/) (2024)
- [EEG-to-fMRI synthesis of task-evoked and spontaneous brain activity: addressing issues of statistical significance and generalizability](https://mlsanity.com/paper/2504.10752/) (2025)
- [A New Strategy for Artificial Intelligence: Training Foundation Models Directly on Human Brain Data](https://mlsanity.com/paper/2601.12053/) (2026)
- [Modeling Spatiotemporal Neural Frames for High Resolution Brain Dynamic](https://mlsanity.com/paper/2603.24176/) (2026)
- [Itextsuperscript2RiMA: Spectral Riemannian Representation with Temporal Attention for Mental Stress Detection based on EEG Signals](https://mlsanity.com/paper/2607.01279/) (2026)
- [MEL: Coordinate-Preserving EEG Tokenization for fMRI Translation](https://mlsanity.com/paper/2608.29304/) (2026)

## Abstract (arXiv)

Functional magnetic resonance imaging (fMRI) is an indispensable tool in modern neuroscience, providing a non-invasive window into whole-brain dynamics at millimeter-scale spatial resolution. However, fMRI is constrained by issues such as high operation costs and immobility. With the rapid advancements in cross-modality synthesis and brain decoding, the use of deep neural networks has emerged as a promising solution for inferring whole-brain, high-resolution fMRI features directly from electroencephalography (EEG), a more widely accessible and portable neuroimaging modality. Nonetheless, the complex projection from neural activity to fMRI hemodynamic responses and the spatial ambiguity of EEG pose substantial challenges both in modeling and interpretability. Relatively few studies to date have developed approaches for EEG-fMRI translation, and although they have made significant strides, the inference of fMRI signals in a given study has been limited to a small set of brain areas and to a single condition (i.e., either resting-state or a specific task). The capability to predict fMRI signals in other brain areas, as well as to generalize across conditions, remain critical gaps in the field. To tackle these challenges, we introduce a novel and generalizable framework: NeuroBOLT, i.e., Neuro-to-BOLD Transformer, which leverages multi-dimensional representation learning from temporal, spatial, and spectral domains to translate raw EEG data to the corresponding fMRI activity signals across the brain. Our experiments demonstrate that NeuroBOLT effectively reconstructs unseen resting-state fMRI signals from primary sensory, high-level cognitive areas, and deep subcortical brain regions, achieving state-of-the-art accuracy with the potential to generalize across varying conditions and sites, which significantly advances the integration of these two modalities.