# Optimizing Memory Efficiency for Deep Convolutional Neural Networks on GPUs

arXiv: https://arxiv.org/abs/1610.03618 · HTML: https://mlsanity.com/paper/1610.03618/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Oct 2016.

## Built on by (most cited first, 7 of 7)

- [A Data-Centric Optimization Framework for Machine Learning](https://mlsanity.com/paper/2110.10802/) (2021)
- [cuConv: A CUDA Implementation of Convolution for CNN Inference](https://mlsanity.com/paper/2103.16234/) (2021)
- [High Performance Im2win and Direct Convolutions using Three Tensor Layouts on SIMD Architectures](https://mlsanity.com/paper/2408.00278/) (2024)
- [Enabling Memory-efficient Im2win Convolution with Multi-precision Support on GPU CUDA and Tensor Cores](https://mlsanity.com/paper/2608.20725/) (2026)
- [ALT: Boosting Deep Learning Performance by Breaking the Wall between Graph and Operator Level Optimizations](https://mlsanity.com/paper/2210.12415/) (2022)
- [Optimal Kernel Tuning Parameter Prediction using Deep Sequence Models](https://mlsanity.com/paper/2404.10162/) (2024)
- [A Configurable and Efficient Memory Hierarchy for Neural Network Hardware Accelerator](https://mlsanity.com/paper/2404.15823/) (2024)

## Abstract (arXiv)

Leveraging large data sets, deep Convolutional Neural Networks (CNNs) achieve state-of-the-art recognition accuracy. Due to the substantial compute and memory operations, however, they require significant execution time. The massive parallel computing capability of GPUs make them as one of the ideal platforms to accelerate CNNs and a number of GPU-based CNN libraries have been developed. While existing works mainly focus on the computational efficiency of CNNs, the memory efficiency of CNNs have been largely overlooked. Yet CNNs have intricate data structures and their memory behavior can have significant impact on the performance. In this work, we study the memory efficiency of various CNN layers and reveal the performance implication from both data layouts and memory access patterns. Experiments show the universal effect of our proposed optimizations on both single layers and various networks, with up to 27.9x for a single layer and up to 5.6x on the whole networks.