# Temporal Conditioning Spiking Latent Variable Models of the Neural Response to Natural Visual Scenes

arXiv: https://arxiv.org/abs/2306.12045 · HTML: https://mlsanity.com/paper/2306.12045/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 89 references that are in the map.
- First posted on arXiv 21 Jun 2023.

## Builds on (most cited first, 29 of 29)

- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks](https://mlsanity.com/paper/1706.02609/) (2017)
- [Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioning](https://mlsanity.com/paper/2208.04202/) (2022)
- [A Recurrent Latent Variable Model for Sequential Data](https://mlsanity.com/paper/1506.02216/) (2015)
- [DRAW: A Recurrent Neural Network For Image Generation](https://mlsanity.com/paper/1502.04623/) (2015)
- [SuperSpike: Supervised learning in multi-layer spiking neural networks](https://mlsanity.com/paper/1705.11146/) (2017)
- [Dive into Deep Learning](https://mlsanity.com/paper/2106.11342/) (2021)
- [Temporal Spike Sequence Learning via Backpropagation for Deep Spiking Neural Networks](https://mlsanity.com/paper/2002.10085/) (2020)
- [Learning Stochastic Recurrent Networks](https://mlsanity.com/paper/1411.7610/) (2014)
- [Variational Recurrent Auto-Encoders](https://mlsanity.com/paper/1412.6581/) (2014)
- [Learning identifiable and interpretable latent models of high-dimensional neural activity using pi-VAE](https://mlsanity.com/paper/2011.04798/) (2020)
- [The neuroconnectionist research programme](https://mlsanity.com/paper/2209.03718/) (2022)
- [Deep Learning Models of the Retinal Response to Natural Scenes](https://mlsanity.com/paper/1702.01825/) (2017)
- [Artificial neural networks for neuroscientists: A primer](https://mlsanity.com/paper/2006.01001/) (2020)
- [Fully Spiking Variational Autoencoder](https://mlsanity.com/paper/2110.00375/) (2021)
- [Targeted Neural Dynamical Modeling](https://mlsanity.com/paper/2110.14853/) (2021)
- [Drop, Swap, and Generate: A Self-Supervised Approach for Generating Neural Activity](https://mlsanity.com/paper/2111.02338/) (2021)
- [Exploiting Noise as a Resource for Computation and Learning in Spiking Neural Networks](https://mlsanity.com/paper/2305.16044/) (2023)
- [Relevant sparse codes with variational information bottleneck](https://mlsanity.com/paper/1605.07332/) (2016)
- [Rescuing neural spike train models from bad MLE](https://mlsanity.com/paper/2010.12362/) (2020)
- [Fitting summary statistics of neural data with a differentiable spiking network simulator](https://mlsanity.com/paper/2106.10064/) (2021)
- [Spike-inspired Rank Coding for Fast and Accurate Recurrent Neural Networks](https://mlsanity.com/paper/2110.02865/) (2021)
- [Kernel methods on spike train space for neuroscience: a tutorial](https://mlsanity.com/paper/1302.5964/) (2013)
- [A new inference approach for training shallow and deep generalized linear models of noisy interacting neurons](https://mlsanity.com/paper/2006.06497/) (2020)

## Built on by (most cited first, 4 of 4)

- [Latent Diffusion for Neural Spiking Data](https://mlsanity.com/paper/2407.08751/) (2024)
- [Revisiting Multivariate Time Series Forecasting with Missing Values](https://mlsanity.com/paper/2509.23494/) (2025)
- [Toward Efficient Deep Spiking Neuron Networks:A Survey On Compression](https://mlsanity.com/paper/2407.08744/) (2024)
- [Aligning Neuronal Coding of Dynamic Visual Scenes with Foundation Vision Models](https://mlsanity.com/paper/2407.10737/) (2024)

## Abstract (arXiv)

Developing computational models of neural response is crucial for understanding sensory processing and neural computations. Current state-of-the-art neural network methods use temporal filters to handle temporal dependencies, resulting in an unrealistic and inflexible processing paradigm. Meanwhile, these methods target trial-averaged firing rates and fail to capture important features in spike trains. This work presents the temporal conditioning spiking latent variable models (TeCoS-LVM) to simulate the neural response to natural visual stimuli. We use spiking neurons to produce spike outputs that directly match the recorded trains. This approach helps to avoid losing information embedded in the original spike trains. We exclude the temporal dimension from the model parameter space and introduce a temporal conditioning operation to allow the model to adaptively explore and exploit temporal dependencies in stimuli sequences in a it natural paradigm. We show that TeCoS-LVM models can produce more realistic spike activities and accurately fit spike statistics than powerful alternatives. Additionally, learned TeCoS-LVM models can generalize well to longer time scales. Overall, while remaining computationally tractable, our model effectively captures key features of neural coding systems. It thus provides a useful tool for building accurate predictive computational accounts for various sensory perception circuits.