# Fully Spiking Variational Autoencoder

arXiv: https://arxiv.org/abs/2110.00375 · HTML: https://mlsanity.com/paper/2110.00375/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 46 references that are in the map.
- First posted on arXiv 26 Sep 2021.

## Builds on (most cited first, 21 of 21)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks](https://mlsanity.com/paper/1506.03099/) (2015)
- [NVAE: A Deep Hierarchical Variational Autoencoder](https://mlsanity.com/paper/2007.03898/) (2020)
- [A Recurrent Latent Variable Model for Sequential Data](https://mlsanity.com/paper/1506.02216/) (2015)
- [Going Deeper With Directly-Trained Larger Spiking Neural Networks](https://mlsanity.com/paper/2011.05280/) (2020)
- [Direct Training for Spiking Neural Networks: Faster, Larger, Better](https://mlsanity.com/paper/1809.05793/) (2018)
- [SuperSpike: Supervised learning in multi-layer spiking neural networks](https://mlsanity.com/paper/1705.11146/) (2017)
- [Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detection](https://mlsanity.com/paper/1903.06530/) (2019)
- [Temporal Spike Sequence Learning via Backpropagation for Deep Spiking Neural Networks](https://mlsanity.com/paper/2002.10085/) (2020)
- [Spike-FlowNet: Event-based Optical Flow Estimation with Energy-Efficient Hybrid Neural Networks](https://mlsanity.com/paper/2003.06696/) (2020)
- [On Aliased Resizing and Surprising Subtleties in GAN Evaluation](https://mlsanity.com/paper/2104.11222/) (2021)
- [Learning to Time-Decode in Spiking Neural Networks Through the Information Bottleneck](https://mlsanity.com/paper/2106.01177/) (2021)
- [Spiking-GAN: A Spiking Generative Adversarial Network Using Time-To-First-Spike Coding](https://mlsanity.com/paper/2106.15420/) (2021)
- [Rescuing neural spike train models from bad MLE](https://mlsanity.com/paper/2010.12362/) (2020)
- [A New Learning Method for Inference Accuracy, Core Occupation, and Performance Co-optimization on TrueNorth Chip](https://mlsanity.com/paper/1604.00697/) (2016)

## Built on by (most cited first, 22 of 22)

- [TCJA-SNN: Temporal-Channel Joint Attention for Spiking Neural Networks](https://mlsanity.com/paper/2206.10177/) (2022)
- [Spiking Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2306.17046/) (2023)
- [Autonomous Driving with Spiking Neural Networks](https://mlsanity.com/paper/2405.19687/) (2024)
- [Poisson Variational Autoencoder](https://mlsanity.com/paper/2405.14473/) (2024)
- [Tensor Decomposition Based Attention Module for Spiking Neural Networks](https://mlsanity.com/paper/2310.14576/) (2023)
- [Spiking-Diffusion: Vector Quantized Discrete Diffusion Model with Spiking Neural Networks](https://mlsanity.com/paper/2308.10187/) (2023)
- [eVAE: Evolutionary Variational Autoencoder](https://mlsanity.com/paper/2301.00011/) (2023)
- [Temporal Conditioning Spiking Latent Variable Models of the Neural Response to Natural Visual Scenes](https://mlsanity.com/paper/2306.12045/) (2023)
- [SDiT: Spiking Diffusion Model with Transformer](https://mlsanity.com/paper/2402.11588/) (2024)
- [SpikeVideoFormer: An Efficient Spike-Driven Video Transformer with Hamming Attention and O(T) Complexity](https://mlsanity.com/paper/2505.10352/) (2025)
- [Spiking Diffusion Models](https://mlsanity.com/paper/2408.16467/) (2024)
- [EA: An Event Autoencoder for High-Speed Vision Sensing](https://mlsanity.com/paper/2507.06459/) (2025)
- [ESVAE: An Efficient Spiking Variational Autoencoder with Reparameterizable Poisson Spiking Sampling](https://mlsanity.com/paper/2310.14839/) (2023)
- [Fully Spiking Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2312.01742/) (2023)
- [Temporal Reversal Regularization for Spiking Neural Networks: Hybrid Spatio-Temporal Invariance for Generalization](https://mlsanity.com/paper/2408.09108/) (2024)
- [Edge Intelligence with Spiking Neural Networks](https://mlsanity.com/paper/2507.14069/) (2025)
- [Negative Binomial Variational Autoencoders for Overdispersed Latent Modeling](https://mlsanity.com/paper/2508.05423/) (2025)
- [Spiking Variational Graph Auto-Encoders for Efficient Graph Representation Learning](https://mlsanity.com/paper/2211.01952/) (2022)
- [Predictive Temporal Attention on Event-based Video Stream for Energy-efficient Situation Awareness](https://mlsanity.com/paper/2402.08936/) (2024)
- [Topology Optimization of Random Memristors for Input-Aware Dynamic SNN](https://mlsanity.com/paper/2407.18625/) (2024)
- [Biologically Inspired Spiking Diffusion Model with Adaptive Lateral Selection Mechanism](https://mlsanity.com/paper/2503.23767/) (2025)
- [LiteEvent-AE: Lightweight Autoencoder for Event-Based Vision on Low-Latency Energy-Constrained Edge Devices](https://mlsanity.com/paper/2608.21764/) (2026)

## Abstract (arXiv)

Spiking neural networks (SNNs) can be run on neuromorphic devices with ultra-high speed and ultra-low energy consumption because of their binary and event-driven nature. Therefore, SNNs are expected to have various applications, including as generative models being running on edge devices to create high-quality images. In this study, we build a variational autoencoder (VAE) with SNN to enable image generation. VAE is known for its stability among generative models; recently, its quality advanced. In vanilla VAE, the latent space is represented as a normal distribution, and floating-point calculations are required in sampling. However, this is not possible in SNNs because all features must be binary time series data. Therefore, we constructed the latent space with an autoregressive SNN model, and randomly selected samples from its output to sample the latent variables. This allows the latent variables to follow the Bernoulli process and allows variational learning. Thus, we build the Fully Spiking Variational Autoencoder where all modules are constructed with SNN. To the best of our knowledge, we are the first to build a VAE only with SNN layers. We experimented with several datasets, and confirmed that it can generate images with the same or better quality compared to conventional ANNs. The code is available at https://github.com/kamata1729/FullySpikingVAE