# Hybrid Neural Autoencoders for Stimulus Encoding in Visual and Other Sensory Neuroprostheses

arXiv: https://arxiv.org/abs/2205.13623 · HTML: https://mlsanity.com/paper/2205.13623/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 75 references that are in the map.
- First posted on arXiv 26 May 2022.

## Builds on (most cited first, 8 of 8)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Ensemble Adversarial Training: Attacks and Defenses](https://mlsanity.com/paper/1705.07204/) (2017)
- [Universal Style Transfer via Feature Transforms](https://mlsanity.com/paper/1705.08086/) (2017)
- [A Deep Neural Network Surrogate for High-Dimensional Random Partial Differential Equations](https://mlsanity.com/paper/1806.02957/) (2018)
- [Deep Learning-Based Perceptual Stimulus Encoder for Bionic Vision](https://mlsanity.com/paper/2203.05604/) (2022)
- [Deep Learning--Based Scene Simplification for Bionic Vision](https://mlsanity.com/paper/2102.00297/) (2021)

## Built on by (most cited first, 10 of 10)

- [Human-in-the-Loop Optimization for Deep Stimulus Encoding in Visual Prostheses](https://mlsanity.com/paper/2306.13104/) (2023)
- [A Deep Learning-based in silico Framework for Optimization on Retinal Prosthetic Stimulation](https://mlsanity.com/paper/2302.03570/) (2023)
- [Optimizing Retinal Prosthetic Stimuli with Conditional Invertible Neural Networks](https://mlsanity.com/paper/2403.04884/) (2024)
- [Evaluating Deep Human-in-the-Loop Optimization for Retinal Implants Using Sighted Participants](https://mlsanity.com/paper/2502.00177/) (2025)
- [Image-to-Brain Signal Generation for Visual Prosthesis with CLIP Guided Multimodal Diffusion Models](https://mlsanity.com/paper/2509.00787/) (2025)
- [Visual Fixation-Based Retinal Prosthetic Simulation](https://mlsanity.com/paper/2410.11688/) (2024)
- [Fuzzing the brain: Automated stress testing for the safety of ML-driven neurostimulation](https://mlsanity.com/paper/2512.05383/) (2025)
- [Percept-Aware Surgical Planning for Visual Cortical Prostheses with Vascular Avoidance](https://mlsanity.com/paper/2603.00362/) (2026)
- [ViBE: Visual-to-M/EEG Brain Encoding via Spatio-Temporal VAE and Distribution-Aligned Projection](https://mlsanity.com/paper/2604.26218/) (2026)
- [Learning to See via Epiretinal Implant Stimulation in silico with Model-Based Deep Reinforcement Learning](https://mlsanity.com/paper/2606.03118/) (2026)

## Abstract (arXiv)

Sensory neuroprostheses are emerging as a promising technology to restore lost sensory function or augment human capabilities. However, sensations elicited by current devices often appear artificial and distorted. Although current models can predict the neural or perceptual response to an electrical stimulus, an optimal stimulation strategy solves the inverse problem: what is the required stimulus to produce a desired response? Here, we frame this as an end-to-end optimization problem, where a deep neural network stimulus encoder is trained to invert a known and fixed forward model that approximates the underlying biological system. As a proof of concept, we demonstrate the effectiveness of this Hybrid Neural Autoencoder (HNA) in visual neuroprostheses. We find that HNA produces high-fidelity patient-specific stimuli representing handwritten digits and segmented images of everyday objects, and significantly outperforms conventional encoding strategies across all simulated patients. Overall this is an important step towards the long-standing challenge of restoring high-quality vision to people living with incurable blindness and may prove a promising solution for a variety of neuroprosthetic technologies.