# Towards Optimal VPU Compiler Cost Modeling by using Neural Networks to Infer Hardware Performances

arXiv: https://arxiv.org/abs/2205.04586 · HTML: https://mlsanity.com/paper/2205.04586/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 11 references that are in the map.
- First posted on arXiv 9 May 2022.

## Builds on (most cited first, 6 of 6)

- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [Predicting the Computational Cost of Deep Learning Models](https://mlsanity.com/paper/1811.11880/) (2018)
- [The CoRa Tensor Compiler: Compilation for Ragged Tensors with Minimal Padding](https://mlsanity.com/paper/2110.10221/) (2021)
- [An Equivalence of Fully Connected Layer and Convolutional Layer](https://mlsanity.com/paper/1712.01252/) (2017)
- [Searching for Efficient Neural Architectures for On-Device ML on Edge TPUs](https://mlsanity.com/paper/2204.14007/) (2022)

## Built on by (most cited first, 2 of 2)

- [ML-driven Hardware Cost Model for MLIR](https://mlsanity.com/paper/2302.11405/) (2023)
- [Neural Architecture Search for Intel Movidius VPU](https://mlsanity.com/paper/2305.03739/) (2023)

## Abstract (arXiv)

Calculating the most efficient schedule of work in a neural network compiler is a difficult task. There are many parameters to be accounted for that can positively or adversely affect that schedule depending on their configuration - How work is shared between distributed targets, the subdivision of tensors to fit in memory, toggling the enablement of optimizations, etc. Traditionally, neural network compilers determine how to set these values by building a graph of choices and choosing the path with minimal 'cost'. These choices and their corresponding costs are usually determined by an algorithm crafted by engineers with a deep knowledge of the target platform. However, when the amount of options available to a compiler is large, it is very difficult to ensure that these models consistently produce an optimal schedule for all scenarios, whilst still completing compilation in an acceptable timeframe. This paper presents 'VPUNN' - a neural network-based cost model trained on low-level task profiling that consistently outperforms the state-of-the-art cost modeling in Intel's line of VPU processors.