# High Performance Zero-Memory Overhead Direct Convolutions

arXiv: https://arxiv.org/abs/1809.10170 · HTML: https://mlsanity.com/paper/1809.10170/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Sep 2018.

## Built on by (most cited first, 14 of 14)

- [Advancing Direct Convolution using Convolution Slicing Optimization and ISA Extensions](https://mlsanity.com/paper/2303.04739/) (2023)
- [Im2win: An Efficient Convolution Paradigm on GPU](https://mlsanity.com/paper/2306.14316/) (2023)
- [Optimizing data-flow in Binary Neural Networks](https://mlsanity.com/paper/2304.00952/) (2023)
- [Im2win: Memory Efficient Convolution On SIMD Architectures](https://mlsanity.com/paper/2306.14320/) (2023)
- [EcoFlow: Efficient Convolutional Dataflows for Low-Power Neural Network Accelerators](https://mlsanity.com/paper/2202.02310/) (2022)
- [Kernel-Segregated Transpose Convolution Operation](https://mlsanity.com/paper/2209.03704/) (2022)
- [High Performance Im2win and Direct Convolutions using Three Tensor Layouts on SIMD Architectures](https://mlsanity.com/paper/2408.00278/) (2024)
- [Enabling Memory-efficient Im2win Convolution with Multi-precision Support on GPU CUDA and Tensor Cores](https://mlsanity.com/paper/2608.20725/) (2026)
- [ConvBench: A Comprehensive Benchmark for 2D Convolution Primitive Evaluation](https://mlsanity.com/paper/2407.10730/) (2024)
- [A Framework to Enable Algorithmic Design Choice Exploration in DNNs](https://mlsanity.com/paper/2410.08300/) (2024)
- [SMM-Conv: Scalar Matrix Multiplication with Zero Packing for Accelerated Convolution](https://mlsanity.com/paper/2411.15659/) (2024)
- [Benchmarking Deep Learning Convolutions on Energy-constrained CPUs](https://mlsanity.com/paper/2509.26217/) (2025)
- [Power-Performance Characterization of TinyML Systems](https://mlsanity.com/paper/2608.21646/) (2026)
- [Right In-Place (RiP) Convolution: A Simple, General, and Near-Optimal Strategy for Memory-Efficient CNN Inference](https://mlsanity.com/paper/2610.00586/) (2026)

## Abstract (arXiv)

The computation of convolution layers in deep neural networks typically rely on high performance routines that trade space for time by using additional memory (either for packing purposes or required as part of the algorithm) to improve performance. The problems with such an approach are two-fold. First, these routines incur additional memory overhead which reduces the overall size of the network that can fit on embedded devices with limited memory capacity. Second, these high performance routines were not optimized for performing convolution, which means that the performance obtained is usually less than conventionally expected. In this paper, we demonstrate that direct convolution, when implemented correctly, eliminates all memory overhead, and yields performance that is between 10% to 400% times better than existing high performance implementations of convolution layers on conventional and embedded CPU architectures. We also show that a high performance direct convolution exhibits better scaling performance, i.e. suffers less performance drop, when increasing the number of threads.