# First-spike based visual categorization using reward-modulated STDP

arXiv: https://arxiv.org/abs/1705.09132 · HTML: https://mlsanity.com/paper/1705.09132/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 May 2017.

## Built on by (most cited first, 19 of 19)

- [A Synapse-Threshold Synergistic Learning Approach for Spiking Neural Networks](https://mlsanity.com/paper/2206.06129/) (2022)
- [Linear Leaky-Integrate-and-Fire Neuron Model Based Spiking Neural Networks and Its Mapping Relationship to Deep Neural Networks](https://mlsanity.com/paper/2207.04889/) (2022)
- [Spiking Neural Networks and Bio-Inspired Supervised Deep Learning: A Survey](https://mlsanity.com/paper/2307.16235/) (2023)
- [Spike time displacement based error backpropagation in convolutional spiking neural networks](https://mlsanity.com/paper/2108.13621/) (2021)
- [Simple and complex spiking neurons: perspectives and analysis in a simple STDP scenario](https://mlsanity.com/paper/2207.04881/) (2022)
- [An optimised deep spiking neural network architecture without gradients](https://mlsanity.com/paper/2109.12813/) (2021)
- [Neuronal Competition Groups with Supervised STDP for Spike-Based Classification](https://mlsanity.com/paper/2410.17066/) (2024)
- [Three-Factor Learning in Spiking Neural Networks: An Overview of Methods and Trends from a Machine Learning Perspective](https://mlsanity.com/paper/2504.05341/) (2025)
- [A Neuromorphic Architecture for Reinforcement Learning from Real-Valued Observations](https://mlsanity.com/paper/2307.02947/) (2023)
- [Efficient and Flexible Neural Network Training through Layer-wise Feedback Propagation](https://mlsanity.com/paper/2308.12053/) (2023)
- [Augmenting learning in neuro-embodied systems through neurobiological first principles](https://mlsanity.com/paper/2407.04525/) (2024)
- [BioLCNet: Reward-modulated Locally Connected Spiking Neural Networks](https://mlsanity.com/paper/2109.05539/) (2021)
- [Spiking neural networks trained via proxy](https://mlsanity.com/paper/2109.13208/) (2021)
- [Efficient Implementation of a Multi-Layer Gradient-Free Online-Trainable Spiking Neural Network on FPGA](https://mlsanity.com/paper/2305.19468/) (2023)
- [STOP: Spatiotemporal Orthogonal Propagation for Weight-Threshold-Leakage Synergistic Training of Deep Spiking Neural Networks](https://mlsanity.com/paper/2411.11082/) (2024)
- [Advancing Direct Training for Spiking Neural Networks with Circulate-Firing Neurons and Learnable Gradients](https://mlsanity.com/paper/2605.27412/) (2026)
- [Frameworks for SNNs: a Review of Data Science-oriented Software and an Expansion of SpykeTorch](https://mlsanity.com/paper/2302.07624/) (2023)
- [NeuroTrain: Surveying Local Learning Rules for Spiking Neural Networks with an Open Benchmarking Framework](https://mlsanity.com/paper/2605.15058/) (2026)
- [Multi-Depth Temporal Fusion for Feedforward, Locally Trained Spiking Neural Networks](https://mlsanity.com/paper/2609.37047/) (2026)

## Abstract (arXiv)

Reinforcement learning (RL) has recently regained popularity, with major achievements such as beating the European game of Go champion. Here, for the first time, we show that RL can be used efficiently to train a spiking neural network (SNN) to perform object recognition in natural images without using an external classifier. We used a feedforward convolutional SNN and a temporal coding scheme where the most strongly activated neurons fire first, while less activated ones fire later, or not at all. In the highest layers, each neuron was assigned to an object category, and it was assumed that the stimulus category was the category of the first neuron to fire. If this assumption was correct, the neuron was rewarded, i.e. spike-timing-dependent plasticity (STDP) was applied, which reinforced the neuron's selectivity. Otherwise, anti-STDP was applied, which encouraged the neuron to learn something else. As demonstrated on various image datasets (Caltech, ETH-80, and NORB), this reward modulated STDP (R-STDP) approach extracted particularly discriminative visual features, whereas classic unsupervised STDP extracts any feature that consistently repeats. As a result, R-STDP outperformed STDP on these datasets. Furthermore, R-STDP is suitable for online learning, and can adapt to drastic changes such as label permutations. Finally, it is worth mentioning that both feature extraction and classification were done with spikes, using at most one spike per neuron. Thus the network is hardware friendly and energy efficient.