# Heterogeneous Neuronal and Synaptic Dynamics for Spike-Efficient Unsupervised Learning: Theory and Design Principles

arXiv: https://arxiv.org/abs/2302.11618 · HTML: https://mlsanity.com/paper/2302.11618/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 86 references that are in the map.
- First posted on arXiv 22 Feb 2023.

## Builds on (most cited first, 24 of 24)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Tutorial on Bayesian Optimization](https://mlsanity.com/paper/1807.02811/) (2018)
- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [Deep Neural Networks as Gaussian Processes](https://mlsanity.com/paper/1711.00165/) (2017)
- [Direct Training for Spiking Neural Networks: Faster, Larger, Better](https://mlsanity.com/paper/1809.05793/) (2018)
- [A Tail-Index Analysis of Stochastic Gradient Noise in Deep Neural Networks](https://mlsanity.com/paper/1901.06053/) (2019)
- [Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detection](https://mlsanity.com/paper/1903.06530/) (2019)
- [The Heidelberg spiking datasets for the systematic evaluation of spiking neural networks](https://mlsanity.com/paper/1910.07407/) (2019)
- [Temporal Spike Sequence Learning via Backpropagation for Deep Spiking Neural Networks](https://mlsanity.com/paper/2002.10085/) (2020)
- [Hausdorff Dimension, Heavy Tails, and Generalization in Neural Networks](https://mlsanity.com/paper/2006.09313/) (2020)
- [Effective and Efficient Computation with Multiple-timescale Spiking Recurrent Neural Networks](https://mlsanity.com/paper/2005.11633/) (2020)
- [Optimizing the energy consumption of spiking neural networks for neuromorphic applications](https://mlsanity.com/paper/1912.01268/) (2019)
- [A Fully Spiking Hybrid Neural Network for Energy-Efficient Object Detection](https://mlsanity.com/paper/2104.10719/) (2021)
- [Predicting non-linear dynamics by stable local learning in a recurrent spiking neural network](https://mlsanity.com/paper/1702.06463/) (2017)
- [ReStoCNet: Residual Stochastic Binary Convolutional Spiking Neural Network for Memory-Efficient Neuromorphic Computing](https://mlsanity.com/paper/1902.04161/) (2019)
- [Heterogeneous Recurrent Spiking Neural Network for Spatio-Temporal Classification](https://mlsanity.com/paper/2211.04297/) (2022)
- [Deep Time-Delay Reservoir Computing: Dynamics and Memory Capacity](https://mlsanity.com/paper/2006.06322/) (2020)
- [Characterization of Generalizability of Spike Timing Dependent Plasticity trained Spiking Neural Networks](https://mlsanity.com/paper/2105.14677/) (2021)
- [Interacting Hawkes processes with multiplicative inhibition](https://mlsanity.com/paper/2105.10597/) (2021)
- [Modeling networks of spiking neurons as interacting processes with memory of variable length](https://mlsanity.com/paper/1502.06446/) (2015)
- [Spatiotemporal dynamics and reliable computations in recurrent spiking neural networks](https://mlsanity.com/paper/1611.01557/) (2016)
- [Learning in Feedback-driven Recurrent Spiking Neural Networks using full-FORCE Training](https://mlsanity.com/paper/2205.13585/) (2022)
- [Firing Rate Dynamics in Recurrent Spiking Neural Networks with Intrinsic and Network Heterogeneity](https://mlsanity.com/paper/1505.03176/) (2015)
- [Microscopic approach of a time elapsed neural model](https://mlsanity.com/paper/1506.02361/) (2015)

## Built on by (most cited first, 8 of 8)

- [Sparse Spiking Neural Network: Exploiting Heterogeneity in Timescales for Pruning Recurrent SNN](https://mlsanity.com/paper/2403.03409/) (2024)
- [Brain-Inspired Spiking Neural Network for Online Unsupervised Time Series Prediction](https://mlsanity.com/paper/2304.04697/) (2023)
- [Temporal Spiking Neural Networks with Synaptic Delay for Graph Reasoning](https://mlsanity.com/paper/2405.16851/) (2024)
- [Topological Representations of Heterogeneous Learning Dynamics of Recurrent Spiking Neural Networks](https://mlsanity.com/paper/2403.12462/) (2024)
- [Exploiting Heterogeneity in Timescales for Sparse Recurrent Spiking Neural Networks for Energy-Efficient Edge Computing](https://mlsanity.com/paper/2407.06452/) (2024)
- [A Dynamical Systems-Inspired Pruning Strategy for Addressing Oversmoothing in Graph Neural Networks](https://mlsanity.com/paper/2412.07243/) (2024)
- [FLAMES: A Hybrid Spiking-State Space Model for Adaptive Memory Retention in Event-Based Learning](https://mlsanity.com/paper/2504.01257/) (2025)
- [HetSyn: Versatile Timescale Integration in Spiking Neural Networks via Heterogeneous Synapses](https://mlsanity.com/paper/2508.11644/) (2025)

## Abstract (arXiv)

This paper shows that the heterogeneity in neuronal and synaptic dynamics reduces the spiking activity of a Recurrent Spiking Neural Network (RSNN) while improving prediction performance, enabling spike-efficient (unsupervised) learning. We analytically show that the diversity in neurons' integration/relaxation dynamics improves an RSNN's ability to learn more distinct input patterns (higher memory capacity), leading to improved classification and prediction performance. We further prove that heterogeneous Spike-Timing-Dependent-Plasticity (STDP) dynamics of synapses reduce spiking activity but preserve memory capacity. The analytical results motivate Heterogeneous RSNN design using Bayesian optimization to determine heterogeneity in neurons and synapses to improve E, defined as the ratio of spiking activity and memory capacity. The empirical results on time series classification and prediction tasks show that optimized HRSNN increases performance and reduces spiking activity compared to a homogeneous RSNN.