# Sigma Delta Quantized Networks

arXiv: https://arxiv.org/abs/1611.02024 · HTML: https://mlsanity.com/paper/1611.02024/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Nov 2016.

## Built on by (most cited first, 13 of 13)

- [Skip-Convolutions for Efficient Video Processing](https://mlsanity.com/paper/2104.11487/) (2021)
- [Language Modeling on a SpiNNaker 2 Neuromorphic Chip](https://mlsanity.com/paper/2312.09084/) (2023)
- [Event Neural Networks](https://mlsanity.com/paper/2112.00891/) (2021)
- [Accelerating Linear Recurrent Neural Networks for the Edge with Unstructured Sparsity](https://mlsanity.com/paper/2502.01330/) (2025)
- [Training for temporal sparsity in deep neural networks, application in video processing](https://mlsanity.com/paper/2107.07305/) (2021)
- [ResQ: Residual Quantization for Video Perception](https://mlsanity.com/paper/2308.09511/) (2023)
- [Synapse Compression for Event-Based Convolutional-Neural-Network Accelerators](https://mlsanity.com/paper/2112.07019/) (2021)
- [Exploiting Symmetric Temporally Sparse BPTT for Efficient RNN Training](https://mlsanity.com/paper/2312.09391/) (2023)
- [Weight Sparsity Complements Activity Sparsity in Neuromorphic Language Models](https://mlsanity.com/paper/2405.00433/) (2024)
- [Event-Stream Super Resolution using Sigma-Delta Neural Network](https://mlsanity.com/paper/2408.06968/) (2024)
- [Neural Network Optimization for Reinforcement Learning Tasks Using Sparse Computations](https://mlsanity.com/paper/2201.02571/) (2022)
- [Autonomous Reinforcement Learning Robot Control with Intel's Loihi 2 Neuromorphic Hardware](https://mlsanity.com/paper/2512.03911/) (2025)
- [Sense Less, Infer More: Agentic Multimodal Transformers for Edge Medical Intelligence](https://mlsanity.com/paper/2604.10404/) (2026)

## Abstract (arXiv)

Deep neural networks can be obscenely wasteful. When processing video, a convolutional network expends a fixed amount of computation for each frame with no regard to the similarity between neighbouring frames. As a result, it ends up repeatedly doing very similar computations. To put an end to such waste, we introduce Sigma-Delta networks. With each new input, each layer in this network sends a discretized form of its change in activation to the next layer. Thus the amount of computation that the network does scales with the amount of change in the input and layer activations, rather than the size of the network. We introduce an optimization method for converting any pre-trained deep network into an optimally efficient Sigma-Delta network, and show that our algorithm, if run on the appropriate hardware, could cut at least an order of magnitude from the computational cost of processing video data.