# Stochastic Synapses Enable Efficient Brain-Inspired Learning Machines

arXiv: https://arxiv.org/abs/1511.04484 · HTML: https://mlsanity.com/paper/1511.04484/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Nov 2015.

## Built on by (most cited first, 21 of 21)

- [Pruning of Deep Spiking Neural Networks through Gradient Rewiring](https://mlsanity.com/paper/2105.04916/) (2021)
- [Exploring Lottery Ticket Hypothesis in Spiking Neural Networks](https://mlsanity.com/paper/2207.01382/) (2022)
- [Adaptive Sparse Structure Development with Pruning and Regeneration for Spiking Neural Networks](https://mlsanity.com/paper/2211.12219/) (2022)
- [Neural Sampling Machine with Stochastic Synapse allows Brain-like Learning and Inference](https://mlsanity.com/paper/2102.10477/) (2021)
- [LoAS: Fully Temporal-Parallel Dataflow for Dual-Sparse Spiking Neural Networks](https://mlsanity.com/paper/2407.14073/) (2024)
- [Developmental Plasticity-inspired Adaptive Pruning for Deep Spiking and Artificial Neural Networks](https://mlsanity.com/paper/2211.12714/) (2022)
- [Noise-Robust Deep Spiking Neural Networks with Temporal Information](https://mlsanity.com/paper/2104.11169/) (2021)
- [Probabilistic Modeling: Proving the Lottery Ticket Hypothesis in Spiking Neural Network](https://mlsanity.com/paper/2305.12148/) (2023)
- [Brain-Inspired Efficient Pruning: Exploiting Criticality in Spiking Neural Networks](https://mlsanity.com/paper/2311.16141/) (2023)
- [Spatio-Temporal Pruning for Compressed Spiking Large Language Models](https://mlsanity.com/paper/2508.20122/) (2025)
- [Gaining the Sparse Rewards by Exploring Lottery Tickets in Spiking Neural Network](https://mlsanity.com/paper/2309.13302/) (2023)
- [Deep Neuromorphic Networks with Superconducting Single Flux Quanta](https://mlsanity.com/paper/2311.10721/) (2023)
- [Locally Learned Synaptic Dropout for Complete Bayesian Inference](https://mlsanity.com/paper/2111.09780/) (2021)
- [Adaptive Synaptic Failure Enables Sampling from Posterior Predictive Distributions in the Brain](https://mlsanity.com/paper/2210.01691/) (2022)
- [Machine learning using magnetic stochastic synapses](https://mlsanity.com/paper/2303.01886/) (2023)
- [Binary stochasticity enabled highly efficient neuromorphic deep learning achieves better-than-software accuracy](https://mlsanity.com/paper/2304.12866/) (2023)
- [Synaptic Sampling of Neural Networks](https://mlsanity.com/paper/2311.13038/) (2023)
- [Estimation of Energy-dissipation Lower-bounds for Neuromorphic Learning-in-memory](https://mlsanity.com/paper/2402.14878/) (2024)
- [Synaptic Modulation using Interspike Intervals Increases Energy Efficiency of Spiking Neural Networks](https://mlsanity.com/paper/2408.02961/) (2024)
- [Spiking Layer-Adaptive Magnitude-based Pruning](https://mlsanity.com/paper/2603.14946/) (2026)
- [NeuroTrain: Surveying Local Learning Rules for Spiking Neural Networks with an Open Benchmarking Framework](https://mlsanity.com/paper/2605.15058/) (2026)

## Abstract (arXiv)

Recent studies have shown that synaptic unreliability is a robust and sufficient mechanism for inducing the stochasticity observed in cortex. Here, we introduce Synaptic Sampling Machines, a class of neural network models that uses synaptic stochasticity as a means to Monte Carlo sampling and unsupervised learning. Similar to the original formulation of Boltzmann machines, these models can be viewed as a stochastic counterpart of Hopfield networks, but where stochasticity is induced by a random mask over the connections. Synaptic stochasticity plays the dual role of an efficient mechanism for sampling, and a regularizer during learning akin to DropConnect. A local synaptic plasticity rule implementing an event-driven form of contrastive divergence enables the learning of generative models in an on-line fashion. Synaptic sampling machines perform equally well using discrete-timed artificial units (as in Hopfield networks) or continuous-timed leaky integrate & fire neurons. The learned representations are remarkably sparse and robust to reductions in bit precision and synapse pruning: removal of more than 75% of the weakest connections followed by cursory re-learning causes a negligible performance loss on benchmark classification tasks. The spiking neuron-based synaptic sampling machines outperform existing spike-based unsupervised learners, while potentially offering substantial advantages in terms of power and complexity, and are thus promising models for on-line learning in brain-inspired hardware.