# Sliding Window Sum Algorithms for Deep Neural Networks

arXiv: https://arxiv.org/abs/2305.16513 · HTML: https://mlsanity.com/paper/2305.16513/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 18 references that are in the map.
- First posted on arXiv 25 May 2023.

## Builds on (most cited first, 8 of 8)

- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects](https://mlsanity.com/paper/2004.02806/) (2020)
- [Caffe: Convolutional Architecture for Fast Feature Embedding](https://mlsanity.com/paper/1408.5093/) (2014)
- [Parallel Multi Channel Convolution using General Matrix Multiplication](https://mlsanity.com/paper/1704.04428/) (2017)
- [The ARM Scalable Vector Extension](https://mlsanity.com/paper/1803.06185/) (2018)
- [Low-memory GEMM-based convolution algorithms for deep neural networks](https://mlsanity.com/paper/1709.03395/) (2017)
- [Parallel approach to sliding window sums](https://mlsanity.com/paper/1811.10074/) (2018)
- [Efficient and Generic 1D Dilated Convolution Layer for Deep Learning](https://mlsanity.com/paper/2104.08002/) (2021)

## Built on by (most cited first, 1 of 1)

- [Accelerating Machine Learning Primitives on Commodity Hardware](https://mlsanity.com/paper/2310.05218/) (2023)

## Abstract (arXiv)

Sliding window sums are widely used for string indexing, hashing and time series analysis. We have developed a family of the generic vectorized sliding sum algorithms that provide speedup of O(P/w) for window size w and number of processors P. For a sum with a commutative operator the speedup is improved to O(P/log(w)). Even more important, our algorithms exhibit efficient memory access patterns. In this paper we study the application of the sliding sum algorithms to the training and inference of the Deep Neural Networks. We demonstrate how both pooling and convolution primitives could be expressed as sliding sums and evaluated by the compute kernels with the shared structure. We show that the sliding sum convolution kernels are more efficient than the commonly used GEMM kernels on the CPU, and could even outperform their GPU counterparts.