# On the second-order optimization for spiking neural networks

arXiv: https://arxiv.org/abs/2609.29379 · HTML: https://mlsanity.com/paper/2609.29379/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 23 references that are in the map.
- First posted on arXiv 24 Sep 2026.

## Builds on (most cited first, 16 of 16)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Optimizing Neural Networks with Kronecker-factored Approximate Curvature](https://mlsanity.com/paper/1503.05671/) (2015)
- [Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccades](https://mlsanity.com/paper/1507.07629/) (2015)
- [Temporal Efficient Training of Spiking Neural Network via Gradient Re-weighting](https://mlsanity.com/paper/2202.11946/) (2022)
- [SuperSpike: Supervised learning in multi-layer spiking neural networks](https://mlsanity.com/paper/1705.11146/) (2017)
- [Learning Long Sequences in Spiking Neural Networks](https://mlsanity.com/paper/2401.00955/) (2023)
- [AdaFisher: Adaptive Second Order Optimization via Fisher Information](https://mlsanity.com/paper/2405.16397/) (2024)
- [Efficient Approximations of the Fisher Matrix in Neural Networks using Kronecker Product Singular Value Decomposition](https://mlsanity.com/paper/2201.10285/) (2022)
- [A feedback control optimizer for online and hardware-aware training of Spiking Neural Networks](https://mlsanity.com/paper/2602.13261/) (2026)
- [The Geometry of Updates: Fisher Alignment at Vocabulary Scale](https://mlsanity.com/paper/2606.27242/) (2026)
- [Linearized Bregman Iterations for Sparse Spiking Neural Networks](https://mlsanity.com/paper/2603.16462/) (2026)
- [Sharpness-Aware Surrogate Training for On-Sensor Spiking Neural Networks](https://mlsanity.com/paper/2604.09696/) (2026)
- [A2SG:Adaptive and Asymmetric Surrogate Gradients for Training Deep Spiking Neural Networks](https://mlsanity.com/paper/2606.11236/) (2026)

## Abstract (arXiv)

Spiking Neural Networks (SNNs) offer an energy-efficient alternative to conventional neural networks by exploiting sparse, binary spikes, and event-driven computation. However, the training of SNNs remains challenging, as spiking activations create a sharp loss landscape that hinders training, and diagonal-curvature optimizers such as the Adam family may fail to capture this geometry. The extension of curvature-based optimization methods to SNNs is further complicated by the sparse, discrete, and temporally recurrent nature of their underlying dynamics. To address these limitations, we propose SpiKFAX, a second-order optimization method that formulates a computationally tractable, Kronecker-factored approximation of the Fisher information matrix specifically adapted to the structure of SNNs. Empirical evaluation across five architectures and seven datasets demonstrates that SpiKFAX consistently yields improvements in test accuracy and training stability relative to other popular optimizers.