# GPU Tensor Cores for fast Arithmetic Reductions

arXiv: https://arxiv.org/abs/2001.05585 · HTML: https://mlsanity.com/paper/2001.05585/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jan 2020.

## Built on by (most cited first, 4 of 4)

- [EvoEngineer: Mastering Automated CUDA Kernel Code Evolution with Large Language Models](https://mlsanity.com/paper/2510.03760/) (2025)
- [From Tokens to Layers: Redefining Stall-Free Scheduling for MoE Serving with Layered Prefill](https://mlsanity.com/paper/2510.08055/) (2025)
- [Learning from distinctive candidates to optimize reduced-precision convolution program on tensor cores](https://mlsanity.com/paper/2202.06819/) (2022)
- [RedFuser: An Automatic Operator Fusion Framework for Cascaded Reductions on AI Accelerators](https://mlsanity.com/paper/2603.10026/) (2026)

## Abstract (arXiv)

This work proposes a GPU tensor core approach that encodes the arithmetic reduction of n numbers as a set of chained m × m matrix multiply accumulate (MMA) operations executed in parallel by GPU tensor cores. The asymptotic running time of the proposed chained tensor core approach is T(n)=5 logm2n and its speedup is S=dfrac45 log2m2 over the classic O(n log n) parallel reduction algorithm. Experimental performance results show that the proposed reduction method is ~ 3.2 × faster than a conventional GPU reduction implementation, and preserves the numerical precision because the sub-results of each chain of R MMAs is kept as a 32-bit floating point value, before being all reduced into as a final 32-bit result. The chained MMA design allows a flexible configuration of thread-blocks; small thread-blocks of 32 or 128 threads can still achieve maximum performance using a chain of R=4,5 MMAs per block, while large thread-blocks work best with R=1. The results obtained in this work show that tensor cores can indeed provide a significant performance improvement to non-Machine Learning applications such as the arithmetic reduction, which is an integration tool for studying many scientific phenomena.