# BigGAN-based Bayesian reconstruction of natural images from human brain activity

arXiv: https://arxiv.org/abs/2003.06105 · HTML: https://mlsanity.com/paper/2003.06105/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Mar 2020.

## Built on by (most cited first, 10 of 10)

- [Brain-Conditional Multimodal Synthesis: A Survey and Taxonomy](https://mlsanity.com/paper/2401.00430/) (2023)
- [Natural Image Reconstruction from fMRI using Deep Learning: A Survey](https://mlsanity.com/paper/2110.09006/) (2021)
- [More Than Meets the Eye: Self-Supervised Depth Reconstruction From Brain Activity](https://mlsanity.com/paper/2106.05113/) (2021)
- [Brain-IT: Image Reconstruction from fMRI via Brain-Interaction Transformer](https://mlsanity.com/paper/2510.25976/) (2025)
- [See What You See: Self-supervised Cross-modal Retrieval of Visual Stimuli from Brain Activity](https://mlsanity.com/paper/2208.03666/) (2022)
- [Multiscale Voxel Based Decoding For Enhanced Natural Image Reconstruction From Brain Activity](https://mlsanity.com/paper/2205.14177/) (2022)
- [Modeling Visual Hallucination: A Generative Adversarial Network Framework](https://mlsanity.com/paper/2102.08209/) (2021)
- [Using Deep Learning for Visual Decoding and Reconstruction from Brain Activity: A Review](https://mlsanity.com/paper/2108.04169/) (2021)
- [Brain-IT-VQA: From Brain Signals to Answers](https://mlsanity.com/paper/2605.29588/) (2026)
- [Data-driven techniques for translational neuroscience and personalized neuro-health](https://mlsanity.com/paper/2608.13749/) (2026)

## Abstract (arXiv)

In the visual decoding domain, visually reconstructing presented images given the corresponding human brain activity monitored by functional magnetic resonance imaging (fMRI) is difficult, especially when reconstructing viewed natural images. Visual reconstruction is a conditional image generation on fMRI data and thus generative adversarial network (GAN) for natural image generation is recently introduced for this task. Although GAN-based methods have greatly improved, the fidelity and naturalness of reconstruction are still unsatisfactory due to the small number of fMRI data samples and the instability of GAN training. In this study, we proposed a new GAN-based Bayesian visual reconstruction method (GAN-BVRM) that includes a classifier to decode categories from fMRI data, a pre-trained conditional generator to generate natural images of specified categories, and a set of encoding models and evaluator to evaluate generated images. GAN-BVRM employs the pre-trained generator of the prevailing BigGAN to generate masses of natural images, and selects the images that best matches with the corresponding brain activity through the encoding models as the reconstruction of the image stimuli. In this process, the semantic and detailed contents of reconstruction are controlled by decoded categories and encoding models, respectively. GAN-BVRM used the Bayesian manner to avoid contradiction between naturalness and fidelity from current GAN-based methods and thus can improve the advantages of GAN. Experimental results revealed that GAN-BVRM improves the fidelity and naturalness, that is, the reconstruction is natural and similar to the presented image stimuli.