# Advancing Direct Convolution using Convolution Slicing Optimization and ISA Extensions

arXiv: https://arxiv.org/abs/2303.04739 · HTML: https://mlsanity.com/paper/2303.04739/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 41 references that are in the map.
- First posted on arXiv 8 Mar 2023.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size](https://mlsanity.com/paper/1602.07360/) (2016)
- [Caffe: Convolutional Architecture for Fast Feature Embedding](https://mlsanity.com/paper/1408.5093/) (2014)
- [cuDNN: Efficient Primitives for Deep Learning](https://mlsanity.com/paper/1410.0759/) (2014)
- [Ansor: Generating High-Performance Tensor Programs for Deep Learning](https://mlsanity.com/paper/2006.06762/) (2020)
- [Parallel Multi Channel Convolution using General Matrix Multiplication](https://mlsanity.com/paper/1704.04428/) (2017)
- [Compiling ONNX Neural Network Models Using MLIR](https://mlsanity.com/paper/2008.08272/) (2020)
- [MEC: Memory-efficient Convolution for Deep Neural Network](https://mlsanity.com/paper/1706.06873/) (2017)
- [High Performance Zero-Memory Overhead Direct Convolutions](https://mlsanity.com/paper/1809.10170/) (2018)
- [Composable and Modular Code Generation in MLIR: A Structured and Retargetable Approach to Tensor Compiler Construction](https://mlsanity.com/paper/2202.03293/) (2022)
- [The Indirect Convolution Algorithm](https://mlsanity.com/paper/1907.02129/) (2019)
- [A matrix math facility for Power ISA(TM) processors](https://mlsanity.com/paper/2104.03142/) (2021)
- [cuConv: A CUDA Implementation of Convolution for CNN Inference](https://mlsanity.com/paper/2103.16234/) (2021)

## Built on by (most cited first, 3 of 3)

- [High Performance Im2win and Direct Convolutions using Three Tensor Layouts on SIMD Architectures](https://mlsanity.com/paper/2408.00278/) (2024)
- [ConvBench: A Comprehensive Benchmark for 2D Convolution Primitive Evaluation](https://mlsanity.com/paper/2407.10730/) (2024)
- [Using MLIR Transform to Design Sliced Convolution Algorithm](https://mlsanity.com/paper/2511.18222/) (2025)

## Abstract (arXiv)

Convolution is one of the most computationally intensive operations that must be performed for machine-learning model inference. A traditional approach to compute convolutions is known as the Im2Col + BLAS method. This paper proposes SConv: a direct-convolution algorithm based on a MLIR/LLVM code-generation toolchain that can be integrated into machine-learning compilers . This algorithm introduces: (a) Convolution Slicing Analysis (CSA) - a convolution-specific 3D cache-blocking analysis pass that focuses on tile reuse over the cache hierarchy; (b) Convolution Slicing Optimization (CSO) - a code-generation pass that uses CSA to generate a tiled direct-convolution macro-kernel; and (c) Vector-Based Packing (VBP) - an architecture-specific optimized input-tensor packing solution based on vector-register shift instructions for convolutions with unitary stride. Experiments conducted on 393 convolutions from full ONNX-MLIR machine-learning models indicate that the elimination of the Im2Col transformation and the use of fast packing routines result in a total packing time reduction, on full model inference, of 2.0x - 3.9x on Intel x86 and 3.6x - 7.2x on IBM POWER10. The speed-up over an Im2Col + BLAS method based on current BLAS implementations for end-to-end machine-learning model inference is in the range of 9% - 25% for Intel x86 and 10% - 42% for IBM POWER10 architectures. The total convolution speedup for model inference is 12% - 27% on Intel x86 and 26% - 46% on IBM POWER10. SConv also outperforms BLAS GEMM, when computing pointwise convolutions, in more than 83% of the 219 tested instances.