# A Deep Learning-based in silico Framework for Optimization on Retinal Prosthetic Stimulation

arXiv: https://arxiv.org/abs/2302.03570 · HTML: https://mlsanity.com/paper/2302.03570/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 29 references that are in the map.
- First posted on arXiv 7 Feb 2023.

## Builds on (most cited first, 6 of 6)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Unified Theory of Early Visual Representations from Retina to Cortex through Anatomically Constrained Deep CNNs](https://mlsanity.com/paper/1901.00945/) (2019)
- [Hybrid Neural Autoencoders for Stimulus Encoding in Visual and Other Sensory Neuroprostheses](https://mlsanity.com/paper/2205.13623/) (2022)
- [Deep Learning-Based Perceptual Stimulus Encoder for Bionic Vision](https://mlsanity.com/paper/2203.05604/) (2022)
- [Efficient Exploration in Binary and Preferential Bayesian Optimization](https://mlsanity.com/paper/2110.09361/) (2021)

## Built on by (most cited first, 3 of 3)

- [Optimizing Retinal Prosthetic Stimuli with Conditional Invertible Neural Networks](https://mlsanity.com/paper/2403.04884/) (2024)
- [Visual Fixation-Based Retinal Prosthetic Simulation](https://mlsanity.com/paper/2410.11688/) (2024)
- [Learning to See via Epiretinal Implant Stimulation in silico with Model-Based Deep Reinforcement Learning](https://mlsanity.com/paper/2606.03118/) (2026)

## Abstract (arXiv)

We propose a neural network-based framework to optimize the perceptions simulated by the in silico retinal implant model pulse2percept. The overall pipeline consists of a trainable encoder, a pre-trained retinal implant model and a pre-trained evaluator. The encoder is a U-Net, which takes the original image and outputs the stimulus. The pre-trained retinal implant model is also a U-Net, which is trained to mimic the biomimetic perceptual model implemented in pulse2percept. The evaluator is a shallow VGG classifier, which is trained with original images. Based on 10,000 test images from the MNIST dataset, we show that the convolutional neural network-based encoder performs significantly better than the trivial downsampling approach, yielding a boost in the weighted F1-Score by 36.17% in the pre-trained classifier with 6x10 electrodes. With this fully neural network-based encoder, the quality of the downstream perceptions can be fine-tuned using gradient descent in an end-to-end fashion.