# Whetstone: A Method for Training Deep Artificial Neural Networks for Binary Communication

arXiv: https://arxiv.org/abs/1810.11521 · HTML: https://mlsanity.com/paper/1810.11521/

## Key facts

- Cited by 14 papers in the map (data through 13 Oct 2026).
- First posted on arXiv 26 Oct 2018.

## Built on by (most cited first, 14 of 14)

- [The Backpropagation Algorithm Implemented on Spiking Neuromorphic Hardware](https://mlsanity.com/paper/2106.07030/) (2021)
- [Brain-inspired Multilayer Perceptron with Spiking Neurons](https://mlsanity.com/paper/2203.14679/) (2022)
- [Keys to Accurate Feature Extraction Using Residual Spiking Neural Networks](https://mlsanity.com/paper/2111.05955/) (2021)
- [Applications of Spiking Neural Networks in Visual Place Recognition](https://mlsanity.com/paper/2311.13186/) (2023)
- [SpGesture: Source-Free Domain-adaptive sEMG-based Gesture Recognition with Jaccard Attentive Spiking Neural Network](https://mlsanity.com/paper/2405.14398/) (2024)
- [Efficient Deep Spiking Multi-Layer Perceptrons with Multiplication-Free Inference](https://mlsanity.com/paper/2306.12465/) (2023)
- [KLIF: An optimized spiking neuron unit for tuning surrogate gradient slope and membrane potential](https://mlsanity.com/paper/2302.09238/) (2023)
- [Rethinking Pretraining as a Bridge from ANNs to SNNs](https://mlsanity.com/paper/2203.01158/) (2022)
- [Training Quantised Neural Networks with STE Variants: the Additive Noise Annealing Algorithm](https://mlsanity.com/paper/2203.11323/) (2022)
- [A noise based novel strategy for faster SNN training](https://mlsanity.com/paper/2211.05453/) (2022)
- [S3NN: Time Step Reduction of Spiking Surrogate Gradients for Training Energy Efficient Single-Step Spiking Neural Networks](https://mlsanity.com/paper/2201.10879/) (2022)
- [AutoML for neuromorphic computing and application-driven co-design: asynchronous, massively parallel optimization of spiking architectures](https://mlsanity.com/paper/2302.13210/) (2023)
- [Spike-based Neuromorphic Computing for Next-Generation Computer Vision](https://mlsanity.com/paper/2310.09692/) (2023)
- [Synaptic Sampling of Neural Networks](https://mlsanity.com/paper/2311.13038/) (2023)

## Abstract (arXiv)

This paper presents a new technique for training networks for low-precision communication. Targeting minimal communication between nodes not only enables the use of emerging spiking neuromorphic platforms, but may additionally streamline processing conventionally. Low-power and embedded neuromorphic processors potentially offer dramatic performance-per-Watt improvements over traditional von Neumann processors, however programming these brain-inspired platforms generally requires platform-specific expertise which limits their applicability. To date, the majority of artificial neural networks have not operated using discrete spike-like communication. We present a method for training deep spiking neural networks using an iterative modification of the backpropagation optimization algorithm. This method, which we call Whetstone, effectively and reliably configures a network for a spiking hardware target with little, if any, loss in performance. Whetstone networks use single time step binary communication and do not require a rate code or other spike-based coding scheme, thus producing networks comparable in timing and size to conventional ANNs, albeit with binarized communication. We demonstrate Whetstone on a number of image classification networks, describing how the sharpening process interacts with different training optimizers and changes the distribution of activity within the network. We further note that Whetstone is compatible with several non-classification neural network applications, such as autoencoders and semantic segmentation. Whetstone is widely extendable and currently implemented using custom activation functions within the Keras wrapper to the popular TensorFlow machine learning framework.