# Learning from distinctive candidates to optimize reduced-precision convolution program on tensor cores

arXiv: https://arxiv.org/abs/2202.06819 · HTML: https://mlsanity.com/paper/2202.06819/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 21 references that are in the map.
- First posted on arXiv 11 Feb 2022.

## Builds on (most cited first, 11 of 11)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [PACT: Parameterized Clipping Activation for Quantized Neural Networks](https://mlsanity.com/paper/1805.06085/) (2018)
- [Ansor: Generating High-Performance Tensor Programs for Deep Learning](https://mlsanity.com/paper/2006.06762/) (2020)
- [Learning to Optimize Tensor Programs](https://mlsanity.com/paper/1805.08166/) (2018)
- [APNN-TC: Accelerating Arbitrary Precision Neural Networks on Ampere GPU Tensor Cores](https://mlsanity.com/paper/2106.12169/) (2021)
- [Accelerating Sparse Matrix-Matrix Multiplication with GPU Tensor Cores](https://mlsanity.com/paper/2009.14600/) (2020)
- [Automatic Kernel Generation for Volta Tensor Cores](https://mlsanity.com/paper/2006.12645/) (2020)
- [GPU Tensor Cores for fast Arithmetic Reductions](https://mlsanity.com/paper/2001.05585/) (2020)
- [Analyzing GPU Tensor Core Potential for Fast Reductions](https://mlsanity.com/paper/1903.03640/) (2019)

## Built on by (most cited first, 1 of 1)

- [Hardware-Aware Features for CUTLASS Kernel Selection](https://mlsanity.com/paper/2609.35587/) (2026)

## Abstract (arXiv)

Convolution is one of the fundamental operations of deep neural networks with demanding matrix computation. In a graphic processing unit (GPU), Tensor Core is a specialized matrix processing hardware equipped with reduced-precision matrix-multiply-accumulate (MMA) instructions to increase throughput. However, it is challenging to achieve optimal performance since the best scheduling of MMA instructions varies for different convolution sizes. In particular, reduced-precision MMA requires many elements grouped as a matrix operand, seriously limiting data reuse and imposing packing and layout overhead on the schedule. This work proposes an automatic scheduling method of reduced-precision MMA for convolution operation. In this method, we devise a search space that explores the thread tile and warp sizes to increase the data reuse despite a large matrix operand of reduced-precision MMA. The search space also includes options of register-level packing and layout optimization to lesson overhead of handling reduced-precision data. Finally, we propose a search algorithm to find the best schedule by learning from the distinctive candidates. This reduced-precision MMA optimization method is evaluated on convolution operations of popular neural networks to demonstrate substantial speedup on Tensor Core compared to the state of the arts with shortened search time.