# Understanding Cache Boundness of ML Operators on ARM Processors

arXiv: https://arxiv.org/abs/2102.00932 · HTML: https://mlsanity.com/paper/2102.00932/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 24 references that are in the map.
- First posted on arXiv 1 Feb 2021.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [Rethinking the Value of Network Pruning](https://mlsanity.com/paper/1810.05270/) (2018)
- [cuDNN: Efficient Primitives for Deep Learning](https://mlsanity.com/paper/1410.0759/) (2014)
- [Learning to Optimize Tensor Programs](https://mlsanity.com/paper/1805.08166/) (2018)
- [Relay: A New IR for Machine Learning Frameworks](https://mlsanity.com/paper/1810.00952/) (2018)
- [BISMO: A Scalable Bit-Serial Matrix Multiplication Overlay for Reconfigurable Computing](https://mlsanity.com/paper/1806.08862/) (2018)
- [Resource-Efficient Neural Networks for Embedded Systems](https://mlsanity.com/paper/2001.03048/) (2020)
- [Efficient Execution of Quantized Deep Learning Models: A Compiler Approach](https://mlsanity.com/paper/2006.10226/) (2020)
- [Automating Generation of Low Precision Deep Learning Operators](https://mlsanity.com/paper/1810.11066/) (2018)
- [Agile Autotuning of a Transprecision Tensor Accelerator Overlay for TVM Compiler Stack](https://mlsanity.com/paper/2004.10854/) (2020)
- [Optimizing Bit-Serial Matrix Multiplication for Reconfigurable Computing](https://mlsanity.com/paper/1901.00370/) (2019)

## Built on by (most cited first, 1 of 1)

- [Towards Hardware-Specific Automatic Compression of Neural Networks](https://mlsanity.com/paper/2212.07818/) (2022)

## Abstract (arXiv)

Machine Learning compilers like TVM allow a fast and flexible deployment on embedded CPUs. This enables the use of non-standard operators, which are common in ML compression techniques. However, it is necessary to understand the limitations of typical compute-intense operators in ML workloads to design a proper solution. This is the first in-detail analysis of dense and convolution operators, generated with TVM, that compares to the fundamental hardware limits of embedded ARM processors. Thereby it explains the gap between computational peak performance, theoretical and measured, and real-world state-of-the-art results, created with TVM and openBLAS. Instead, one can see that single-precision general matrix multiply (GEMM) and convolutions are bound by L1-cache-read bandwidth. Explorations of 8-bit and bit-serial quantized operators show that quantization can be used to achieve relevant speedups compared to cache-bound floating-point operators. However, the performance of quantized operators highly depends on the interaction between data layout and bit packing.