# Tensor Processing Primitives: A Programming Abstraction for Efficiency and Portability in Deep Learning & HPC Workloads

arXiv: https://arxiv.org/abs/2104.05755 · HTML: https://mlsanity.com/paper/2104.05755/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 71 references that are in the map.
- First posted on arXiv 12 Apr 2021.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [Deep Learning Recommendation Model for Personalization and Recommendation Systems](https://mlsanity.com/paper/1906.00091/) (2019)
- [cuDNN: Efficient Primitives for Deep Learning](https://mlsanity.com/paper/1410.0759/) (2014)
- [Ansor: Generating High-Performance Tensor Programs for Deep Learning](https://mlsanity.com/paper/2006.06762/) (2020)
- [Tensor Comprehensions: Framework-Agnostic High-Performance Machine Learning Abstractions](https://mlsanity.com/paper/1802.04730/) (2018)
- [The Deep Learning Compiler: A Comprehensive Survey](https://mlsanity.com/paper/2002.03794/) (2020)
- [Deep Learning for Computational Chemistry](https://mlsanity.com/paper/1701.04503/) (2017)
- [A Survey of Deep Learning for Scientific Discovery](https://mlsanity.com/paper/2003.11755/) (2020)
- [Optimizing Deep Learning Recommender Systems' Training On CPU Cluster Architectures](https://mlsanity.com/paper/2005.04680/) (2020)
- [Feature Learning in Deep Neural Networks - Studies on Speech Recognition Tasks](https://mlsanity.com/paper/1301.3605/) (2013)
- [Anatomy Of High-Performance Deep Learning Convolutions On SIMD Architectures](https://mlsanity.com/paper/1808.05567/) (2018)
- [Stripe: Tensor Compilation via the Nested Polyhedral Model](https://mlsanity.com/paper/1903.06498/) (2019)
- [Deep Graph Library Optimizations for Intel(R) x86 Architecture](https://mlsanity.com/paper/2007.06354/) (2020)
- [DBCSR: A Blocked Sparse Tensor Algebra Library](https://mlsanity.com/paper/1910.13555/) (2019)

## Built on by (most cited first, 9 of 9)

- [ML-SpecQD: Multi-Level Speculative Decoding with Quantized Drafts](https://mlsanity.com/paper/2503.13565/) (2025)
- [oneDNN Graph Compiler: A Hybrid Approach for High-Performance Deep Learning Compilation](https://mlsanity.com/paper/2301.01333/) (2023)
- [Generative Active Learning for the Search of Small-molecule Protein Binders](https://mlsanity.com/paper/2405.01616/) (2024)
- [Harnessing Deep Learning and HPC Kernels via High-Level Loop and Tensor Abstractions on CPU Architectures](https://mlsanity.com/paper/2304.12576/) (2023)
- [Towards a high-performance AI compiler with upstream MLIR](https://mlsanity.com/paper/2404.15204/) (2024)
- [DECA: A Near-Core LLM Decompression Accelerator Grounded on a 3D Roofline Model](https://mlsanity.com/paper/2505.19349/) (2025)
- [Pushing the Envelope of LLM Inference with Ultra-Low-Bit Quantized Models](https://mlsanity.com/paper/2508.06753/) (2025)
- [DistGNN-MB: Distributed Large-Scale Graph Neural Network Training on x86 via Minibatch Sampling](https://mlsanity.com/paper/2211.06385/) (2022)
- [Space Filling Curves is All You Need: Communication-Avoiding Matrix Multiplication Made Simple](https://mlsanity.com/paper/2601.16294/) (2026)

## Abstract (arXiv)

During the past decade, novel Deep Learning (DL) algorithms, workloads and hardware have been developed to tackle a wide range of problems. Despite the advances in workload and hardware ecosystems, the programming methodology of DL systems is stagnant. DL workloads leverage either highly-optimized, yet platform-specific and inflexible kernels from DL libraries, or in the case of novel operators, reference implementations are built via DL framework primitives with underwhelming performance. This work introduces the Tensor Processing Primitives (TPP), a programming abstraction striving for efficient, portable implementation of DL workloads with high-productivity. TPPs define a compact, yet versatile set of 2D-tensor operators (or a virtual Tensor ISA), which subsequently can be utilized as building-blocks to construct complex operators on high-dimensional tensors. The TPP specification is platform-agnostic, thus code expressed via TPPs is portable, whereas the TPP implementation is highly-optimized and platform-specific. We demonstrate the efficacy and viability of our approach using standalone kernels and end-to-end DL & HPC workloads expressed entirely via TPPs that outperform state-of-the-art implementations on multiple platforms.