Hi-DREAM: Brain-Inspired Hierarchical Diffusion for fMRI-to-Image Reconstruction via ROI Encoder and VisuAl Mapping
Not on the map. Why?
Abstract (arXiv)
Reconstructing natural images from fMRI requires bridging neural activity with both the structural and semantic representations used by modern generative models. Existing diffusion-based decoders often condition on a single global fMRI embedding, which limits their ability to exploit the hierarchical organization of the visual cortex and makes the contribution of different visual areas difficult to inspect. We propose Hi-DREAM, a brain-inspired hierarchical diffusion framework that structures fMRI conditioning according to early, middle, and late visual Regions of Interest (ROI) streams. A ROI adapter converts these streams into a multi-scale cortical pyramid, and a lightweight ROI-conditioned ControlNet injects the resulting anatomy-aware priors into matched U-Net depths during denoising. Experiments on the Natural Scenes Dataset (NSD) show that Hi-DREAM achieves state-of-the-art high-level semantic reconstruction while retaining strong low-level structure. Further ablation and attribution analyses show that the proposed hierarchy-aware conditioning is effective, and that different ROI streams provide complementary, inspectable contributions to reconstruction.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 24 of its 59 references that are in the map. First posted on arXiv 14 Nov 2025.
Builds on
24 of 59 references.
References · 24
- 2017Attention Is All You Needcited by 60,361
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
- 2022Classifier-Free Diffusion Guidancecited by 5,847
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
- 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
- 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
- 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
- 2024MindEye2: Shared-Subject Models Enable fMRI-To-Image With 1 Hour of Datacited by 109
- 2022Mind Reader: Reconstructing complex images from brain activitiescited by 85
12 more
- 2023Cinematic Mindscapes: High-quality Video Reconstruction from Brain Activitycited by 71
- 2024MindBridge: A Cross-Subject Brain Decoding Frameworkcited by 66
- 2023MindDiffuser: Controlled Image Reconstruction from Human Brain Activity with Semantic and Structural Diffusioncited by 47
- 2023DREAM: Visual Decoding from Reversing Human Visual Systemcited by 44
- 2024Psychometry: An Omnifit Model for Image Reconstruction from Human Brain Activitycited by 22
- 2024NeuralDiffuser: Neuroscience-inspired Diffusion Guidance for fMRI Visual Reconstructioncited by 10
- 2024Uncovering cognitive taskonomy through transfer learning in masked autoencoder-based fMRI reconstructioncited by 7
- 2022A study of deep perceptual metrics for image quality assessmentcited by 5
- 2024Across-subject ensemble-learning alleviates the need for large samples for fMRI decodingcited by 3
- 2024Brain-Streams: fMRI-to-Image Reconstruction with Multi-modal Guidancecited by 3
- 2021Attention module improves both performance and interpretability of 4D fMRI decoding neural networkcited by 2
- 2024An Adaptively Weighted Averaging Method for Regional Time Series Extraction of fMRI-based Brain Decodingcited by 1
Built on by
Not cited yet.