# The Brain-Inspired Decoder for Natural Visual Image Reconstruction

arXiv: https://arxiv.org/abs/2207.08591 · HTML: https://mlsanity.com/paper/2207.08591/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 39 references that are in the map.
- First posted on arXiv 18 Jul 2022.

## Builds on (most cited first, 6 of 6)

- [Understanding the Effective Receptive Field in Deep Convolutional Neural Networks](https://mlsanity.com/paper/1701.04128/) (2017)
- [Generic decoding of seen and imagined objects using hierarchical visual features](https://mlsanity.com/paper/1510.06479/) (2015)
- [Gabor Convolutional Networks](https://mlsanity.com/paper/1705.01450/) (2017)
- [Reconstruction of Natural Visual Scenes from Neural Spikes with Deep Neural Networks](https://mlsanity.com/paper/1904.13007/) (2019)
- [Deep Auto-encoder with Neural Response](https://mlsanity.com/paper/2111.15309/) (2021)
- [Unsupervised decoding of spinal motor neuron spike trains for estimating hand kinematics following targeted muscle reinnervation](https://mlsanity.com/paper/1910.09120/) (2019)

## Built on by (most cited first, 2 of 2)

- [Seeing Beyond the Brain: Conditional Diffusion Model with Sparse Masked Modeling for Vision Decoding](https://mlsanity.com/paper/2211.06956/) (2022)
- [MEIcoder: Decoding Visual Stimuli from Neural Activity by Leveraging Most Exciting Inputs](https://mlsanity.com/paper/2510.20762/) (2025)

## Abstract (arXiv)

Decoding images from brain activity has been a challenge. Owing to the development of deep learning, there are available tools to solve this problem. The decoded image, which aims to map neural spike trains to low-level visual features and high-level semantic information space. Recently, there are a few studies of decoding from spike trains, however, these studies pay less attention to the foundations of neuroscience and there are few studies that merged receptive field into visual image reconstruction. In this paper, we propose a deep learning neural network architecture with biological properties to reconstruct visual image from spike trains. As far as we know, we implemented a method that integrated receptive field property matrix into loss function at the first time. Our model is an end-to-end decoder from neural spike trains to images. We not only merged Gabor filter into auto-encoder which used to generate images but also proposed a loss function with receptive field properties. We evaluated our decoder on two datasets which contain macaque primary visual cortex neural spikes and salamander retina ganglion cells (RGCs) spikes. Our results show that our method can effectively combine receptive field features to reconstruct images, providing a new approach to visual reconstruction based on neural information.