mlsanityData to 7 Oct 2026

Reconstructing Perceptive Images from Brain Activity by Shape-Semantic GAN

arXiv 2101.12083 ↗First posted 28 Jan 2021cs.NEcited by 31

Not on the map. Why?

Abstract (arXiv)

Reconstructing seeing images from fMRI recordings is an absorbing research area in neuroscience and provides a potential brain-reading technology. The challenge lies in that visual encoding in brain is highly complex and not fully revealed. Inspired by the theory that visual features are hierarchically represented in cortex, we propose to break the complex visual signals into multi-level components and decode each component separately. Specifically, we decode shape and semantic representations from the lower and higher visual cortex respectively, and merge the shape and semantic information to images by a generative adversarial network (Shape-Semantic GAN). This 'divide and conquer' strategy captures visual information more accurately. Experiments demonstrate that Shape-Semantic GAN improves the reconstruction similarity and image quality, and achieves the state-of-the-art image reconstruction performance.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 Jan 2021.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 4; 2023: 11; 2024: 4; 2025: 8; 2026: 2.

31 papers.

Sort:

By area:

  1. 2022Seeing Beyond the Brain: Conditional Diffusion Model with Sparse Masked Modeling for Vision Decodingcited by 142
  2. 2022Mind Reader: Reconstructing complex images from brain activitiescited by 85
  3. 2023Decoding Natural Images from EEG for Object Recognitioncited by 79
  4. 2021Multimodal Image Synthesis and Editing: The Generative AI Eracited by 52
  5. 2022Decoding natural image stimuli from fMRI data with a surface-based convolutional networkcited by 48
  6. 2023MindDiffuser: Controlled Image Reconstruction from Human Brain Activity with Semantic and Structural Diffusioncited by 47
  7. 2023UniBrain: Unify Image Reconstruction and Captioning All in One Diffusion Model from Human Brain Activitycited by 42
  8. 2023Contrast, Attend and Diffuse to Decode High-Resolution Images from Brain Activitiescited by 36
  9. 2023BrainVis: Exploring the Bridge between Brain and Visual Signals via Image Reconstructioncited by 30
  10. 2023AI-Generated Content (AIGC) for Various Data Modalities: A Surveycited by 25
  11. 2023Deep Neural Networks and Brain Alignment: Brain Encoding and Decoding (Survey)cited by 20
  12. 2023Brain-Conditional Multimodal Synthesis: A Survey and Taxonomycited by 15
19 more
  1. 2024NeuroCine: Decoding Vivid Video Sequences from Human Brain Activtiescited by 10
  2. 2021Natural Image Reconstruction from fMRI using Deep Learning: A Surveycited by 9
  3. 2025MoRE-Brain: Routed Mixture of Experts for Interpretable and Generalizable Cross-Subject fMRI Visual Decodingcited by 7
  4. 2023MindGPT: Interpreting What You See with Non-invasive Brain Recordingscited by 6
  5. 2025A Survey of fMRI to Image Reconstructioncited by 6
  6. 2025A Survey on fMRI-based Brain Decoding for Reconstructing Multimodal Stimulicited by 6
  7. 2023Decoding Realistic Images from Brain Activity with Contrastive Self-supervision and Latent Diffusioncited by 5
  8. 2024MindSemantix: Deciphering Brain Visual Experiences with a Brain-Language Modelcited by 2
  9. 2022Facial Image Reconstruction from Functional Magnetic Resonance Imaging via GAN Inversion with Improved Attribute Consistencycited by 1
  10. 2024BrainDecoder: Style-Based Visual Decoding of EEG Signalscited by 1
  11. 2025Schr"odinger Diffusion Driven Signal Recovery in 3T BOLD fMRI Using Unmatched 7T Observationscited by 1
  12. 2025MindShot: Multi-Shot Video Reconstruction from fMRI with LLM Decodingcited by 1
  13. 2025HAVIR: HierArchical Vision to Image Reconstruction using CLIP-Guided Versatile Diffusioncited by 1
  14. 2026SynMind: Reducing Semantic Hallucination in fMRI-Based Image Reconstructioncited by 1
  15. 2023MindDiffuser: Controlled Image Reconstruction from Human Brain Activity with Semantic and Structural Diffusioncited by 0
  16. 2024Reconstructing Retinal Visual Images from 3T fMRI Data Enhanced by Unsupervised Learningcited by 0
  17. 2025HAVIR: HierArchical Vision to Image Reconstruction using CLIP-Guided Versatile Diffusioncited by 0
  18. 2025NeuroSwift: A Lightweight Cross-Subject Framework for fMRI Visual Reconstruction of Complex Scenescited by 0
  19. 2026STSBench: A Large-Scale Dataset for Modeling Neuronal Activity in the Dorsal Stream of Primate Visual Cortexcited by 0