# Bayesian Continual Learning via Spiking Neural Networks

arXiv: https://arxiv.org/abs/2208.13723 · HTML: https://mlsanity.com/paper/2208.13723/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 91 references that are in the map.
- First posted on arXiv 29 Aug 2022.

## Builds on (most cited first, 30 of 35)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images](https://mlsanity.com/paper/1412.1897/) (2014)
- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [SLAYER: Spike Layer Error Reassignment in Time](https://mlsanity.com/paper/1810.08646/) (2018)
- [Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks](https://mlsanity.com/paper/2002.10118/) (2020)
- [SuperSpike: Supervised learning in multi-layer spiking neural networks](https://mlsanity.com/paper/1705.11146/) (2017)
- [Practical Deep Learning with Bayesian Principles](https://mlsanity.com/paper/1906.02506/) (2019)
- [Towards Robust Evaluations of Continual Learning](https://mlsanity.com/paper/1805.09733/) (2018)
- [A Primer on PAC-Bayesian Learning](https://mlsanity.com/paper/1901.05353/) (2019)
- [Uncertainty-guided Continual Learning with Bayesian Neural Networks](https://mlsanity.com/paper/1906.02425/) (2019)
- [The Bayesian Learning Rule](https://mlsanity.com/paper/2107.04562/) (2021)
- [Eligibility Traces and Plasticity on Behavioral Time Scales: Experimental Support of neoHebbian Three-Factor Learning Rules](https://mlsanity.com/paper/1801.05219/) (2018)
- [Conjugate-Computation Variational Inference : Converting Variational Inference in Non-Conjugate Models to Inferences in Conjugate Models](https://mlsanity.com/paper/1703.04265/) (2017)
- [Gradient Descent for Spiking Neural Networks](https://mlsanity.com/paper/1706.04698/) (2017)
- [A Unifying Bayesian View of Continual Learning](https://mlsanity.com/paper/1902.06494/) (2019)
- [Bayesian Variational Autoencoders for Unsupervised Out-of-Distribution Detection](https://mlsanity.com/paper/1912.05651/) (2019)
- [Synaptic metaplasticity in binarized neural networks](https://mlsanity.com/paper/2101.07592/) (2021)
- [An Introduction to Probabilistic Spiking Neural Networks: Probabilistic Models, Learning Rules, and Applications](https://mlsanity.com/paper/1910.01059/) (2019)
- [Patterns of Scalable Bayesian Inference](https://mlsanity.com/paper/1602.05221/) (2016)
- [Training Binary Neural Networks using the Bayesian Learning Rule](https://mlsanity.com/paper/2002.10778/) (2020)
- [lpSpikeCon: Enabling Low-Precision Spiking Neural Network Processing for Efficient Unsupervised Continual Learning on Autonomous Agents](https://mlsanity.com/paper/2205.12295/) (2022)
- [Federated Neuromorphic Learning of Spiking Neural Networks for Low-Power Edge Intelligence](https://mlsanity.com/paper/1910.09594/) (2019)
- [Learning to Time-Decode in Spiking Neural Networks Through the Information Bottleneck](https://mlsanity.com/paper/2106.01177/) (2021)
- [Network Plasticity as Bayesian Inference](https://mlsanity.com/paper/1504.05143/) (2015)
- [Memristors -- from In-memory computing, Deep Learning Acceleration, Spiking Neural Networks, to the Future of Neuromorphic and Bio-inspired Computing](https://mlsanity.com/paper/2004.14942/) (2020)
- [BiSNN: Training Spiking Neural Networks with Binary Weights via Bayesian Learning](https://mlsanity.com/paper/2012.08300/) (2020)

## Built on by (most cited first, 13 of 13)

- [Knowing When to Stop: Delay-Adaptive Spiking Neural Network Classifiers with Reliability Guarantees](https://mlsanity.com/paper/2305.11322/) (2023)
- [Continual Learning with Neuromorphic Computing: Foundations, Methods, and Emerging Applications](https://mlsanity.com/paper/2410.09218/) (2024)
- [Bayesian Inference on Binary Spiking Networks Leveraging Nanoscale Device Stochasticity](https://mlsanity.com/paper/2302.01302/) (2023)
- [Compressed Latent Replays for Lightweight Continual Learning on Spiking Neural Networks](https://mlsanity.com/paper/2407.03111/) (2024)
- [Replay4NCL: An Efficient Memory Replay-based Methodology for Neuromorphic Continual Learning in Embedded AI Systems](https://mlsanity.com/paper/2503.17061/) (2025)
- [Online Continual Learning via Spiking Neural Networks with Sleep Enhanced Latent Replay](https://mlsanity.com/paper/2507.02901/) (2025)
- [Stochastic Quantum Spiking Neural Networks with Quantum Memory and Local Learning](https://mlsanity.com/paper/2506.21324/) (2025)
- [Towards Improved Imbalance Robustness in Continual Multi-Label Learning with Dual Output Spiking Architecture (DOSA)](https://mlsanity.com/paper/2402.04596/) (2024)
- [Investigating Plausibility of Biologically Inspired Bayesian Learning in ANNs](https://mlsanity.com/paper/2411.18788/) (2024)
- [Training of Spiking Neural Networks with Expectation-Propagation](https://mlsanity.com/paper/2506.23757/) (2025)
- [Multi-Scale Temporal Homeostasis Enables Efficient and Robust Neural Networks](https://mlsanity.com/paper/2602.07009/) (2026)
- [Practical Bayesian Inference for Speech SNNs: Uncertainty and Loss-Landscape Smoothing](https://mlsanity.com/paper/2604.08624/) (2026)
- [Saliency-Guided Replay for Continual Learning in Deep Spiking Neural Networks](https://mlsanity.com/paper/2610.17998/) (2026)

## Abstract (arXiv)

Among the main features of biological intelligence are energy efficiency, capacity for continual adaptation, and risk management via uncertainty quantification. Neuromorphic engineering has been thus far mostly driven by the goal of implementing energy-efficient machines that take inspiration from the time-based computing paradigm of biological brains. In this paper, we take steps towards the design of neuromorphic systems that are capable of adaptation to changing learning tasks, while producing well-calibrated uncertainty quantification estimates. To this end, we derive online learning rules for spiking neural networks (SNNs) within a Bayesian continual learning framework. In it, each synaptic weight is represented by parameters that quantify the current epistemic uncertainty resulting from prior knowledge and observed data. The proposed online rules update the distribution parameters in a streaming fashion as data are observed. We instantiate the proposed approach for both real-valued and binary synaptic weights. Experimental results using Intel's Lava platform show the merits of Bayesian over frequentist learning in terms of capacity for adaptation and uncertainty quantification.