# Characterizing and Demystifying the Implicit Convolution Algorithm on Commercial Matrix-Multiplication Accelerators

arXiv: https://arxiv.org/abs/2110.03901 · HTML: https://mlsanity.com/paper/2110.03901/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Oct 2021.

## Built on by (most cited first, 8 of 8)

- [OliVe: Accelerating Large Language Models via Hardware-friendly Outlier-Victim Pair Quantization](https://mlsanity.com/paper/2304.07493/) (2023)
- [ANT: Exploiting Adaptive Numerical Data Type for Low-bit Deep Neural Network Quantization](https://mlsanity.com/paper/2208.14286/) (2022)
- [M-ANT: Efficient Low-bit Group Quantization for LLMs via Mathematically Adaptive Numerical Type](https://mlsanity.com/paper/2502.18755/) (2025)
- [CuBridge: An LLM-Based Framework for Understanding and Reconstructing High-Performance Attention Kernels](https://mlsanity.com/paper/2605.05023/) (2026)
- [AdaptGear: Accelerating GNN Training via Adaptive Subgraph-Level Kernels on GPUs](https://mlsanity.com/paper/2305.17408/) (2023)
- [Performance Analysis of DNN Inference/Training with Convolution and non-Convolution Operations](https://mlsanity.com/paper/2306.16767/) (2023)
- [Benchmarking Deep Learning Convolutions on Energy-constrained CPUs](https://mlsanity.com/paper/2509.26217/) (2025)
- [MicroQonv: Reshaping Convolution Tensors for Efficient Microscaling in Training and Inference](https://mlsanity.com/paper/2609.28358/) (2026)

## Abstract (arXiv)

Many of today's deep neural network accelerators, e.g., Google's TPU and NVIDIA's tensor core, are built around accelerating the general matrix multiplication (i.e., GEMM). However, supporting convolution on GEMM-based accelerators is not trivial. The naive method explicitly lowers the convolution to GEMM, commonly known as im2col, which introduces significant performance and memory overhead. Existing implicit im2col algorithms require unscalable hardware and are inefficient in supporting important convolution variants such as strided convolution. In this paper, we propose a memory-efficient and hardware-friendly implicit im2col algorithm used by Google's TPU, which dynamically converts a convolution into a GEMM with practically zero performance and memory overhead, fully unleashing the power of GEMM engines. Through comprehensive experimental results, we quantitatively argue that this algorithm has been adopted in commercial closed-source platforms, and we are the first to describe its high-level idea and implementation details. Finally, we show that our algorithm can also be generally applied to Nvidia's Tensor Cores (TC), matching and out-performing the measured performance on TCs.