mlsanityData to 14 Oct 2026

HAVIR: HierArchical Vision to Image Reconstruction using CLIP-Guided Versatile Diffusion

arXiv 2510.03122 ↗First posted 3 Oct 2025cs.CV cs.AIcited by 1

On the map: Diffusion models → Medical imaging and segmentation

Abstract (arXiv)

The reconstruction of visual information from brain activity fosters interdisciplinary integration between neuroscience and computer vision. However, existing methods still face challenges in accurately recovering highly complex visual stimuli. This difficulty stems from the characteristics of natural scenes: low-level features exhibit heterogeneity, while high-level features show semantic entanglement due to contextual overlaps. Inspired by the hierarchical representation theory of the visual cortex, we propose the HAVIR model, which separates the visual cortex into two hierarchical regions and extracts distinct features from each. Specifically, the Structural Generator extracts structural information from spatial processing voxels and converts it into latent diffusion priors, while the Semantic Extractor converts semantic processing voxels into CLIP embeddings. These components are integrated via the Versatile Diffusion model to synthesize the final image. Experimental results demonstrate that HAVIR enhances both the structural and semantic quality of reconstructions, even in complex scenes, and outperforms existing models.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models. Builds on 27 of its 50 references that are in the map. First posted on arXiv 3 Oct 2025.

Builds on

27 of 50 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024NeuroPictor: Refining fMRI-to-Image Reconstruction via Multi-individual Pretraining and Multi-level Modulation27 Mar
  2. 2023UniBrain: Unify Image Reconstruction and Captioning All in One Diffusion Model from Human Brain Activity14 Aug
  3. 2023Reconstructing the Mind's Eye: fMRI-to-Image with Contrastive Learning and Diffusion Priors29 May
  4. 2023Natural scene reconstruction from fMRI signals using generative latent diffusion9 Mar

Other references · 23

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  5. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  6. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  7. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  8. 2021Palette: Image-to-Image Diffusion Modelscited by 1,300
  9. 2015Holistically-Nested Edge Detectioncited by 592
  10. 2023Patch Diffusion: Faster and More Data-Efficient Training of Diffusion Modelscited by 133
  11. 2015Generic decoding of seen and imagined objects using hierarchical visual featurescited by 120
  12. 2022Mind Reader: Reconstructing complex images from brain activitiescited by 85
11 more
  1. 2019From voxels to pixels and back: Self-supervision in natural-image reconstruction from fMRIcited by 67
  2. 2024MindBridge: A Cross-Subject Brain Decoding Frameworkcited by 66
  3. 2022Decoding natural image stimuli from fMRI data with a surface-based convolutional networkcited by 49
  4. 2023MindDiffuser: Controlled Image Reconstruction from Human Brain Activity with Semantic and Structural Diffusioncited by 47
  5. 2020Reconstructing Natural Scenes from fMRI Patterns using BigBiGANcited by 44
  6. 2023DREAM: Visual Decoding from Reversing Human Visual Systemcited by 44
  7. 2018Reconstructing Faces from fMRI Patterns using Deep Generative Neural Networkscited by 43
  8. 2021Reconstructing Perceptive Images from Brain Activity by Shape-Semantic GANcited by 32
  9. 2023Stable Diffusion is Unstablecited by 27
  10. 2024Psychometry: An Omnifit Model for Image Reconstruction from Human Brain Activitycited by 22
  11. 2025BrainGuard: Privacy-Preserving Multisubject Image Reconstructions from Brain Activitiescited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026HyNeuralMap: Hyperbolic Mapping of Visual Semantics to Neural Hierarchiescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025HAVIR: HierArchical Vision to Image Reconstruction using CLIP-Guided Versatile Diffusion
  2. 2025Neurons: Emulating the Human Visual Cortex Improves Fidelity and Interpretability in fMRI-to-Video Reconstruction
  3. 2024Brain-Streams: fMRI-to-Image Reconstruction with Multi-modal Guidance
  4. 2022Semantic Brain Decoding: from fMRI to conceptually similar image reconstruction of visual stimuli
  5. 2023Reconstructing seen images from human brain activity via guided stochastic search
  6. 2026Versatile Framework with Semantic and Structural guidance for Image Reconstruction from Brain Activity
  7. 2024NeuralDiffuser: Neuroscience-inspired Diffusion Guidance for fMRI Visual Reconstruction
  8. 2025NeuroSwift: A Lightweight Cross-Subject Framework for fMRI Visual Reconstruction of Complex Scenes