# HEP-BNN: A Framework for Finding Low-Latency Execution Configurations of BNNs on Heterogeneous Multiprocessor Platforms

arXiv: https://arxiv.org/abs/2301.05126 · HTML: https://mlsanity.com/paper/2301.05126/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 11 references that are in the map.
- First posted on arXiv 12 Jan 2023.

## Builds on (most cited first, 6 of 6)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Accelerating Binarized Neural Networks via Bit-Tensor-Cores in Turing GPUs](https://mlsanity.com/paper/2006.16578/) (2020)
- [How Does Batch Normalization Help Binary Training?](https://mlsanity.com/paper/1909.09139/) (2019)
- [PhoneBit: Efficient GPU-Accelerated Binary Neural Network Inference Engine for Mobile Phones](https://mlsanity.com/paper/1912.04050/) (2019)
- [A Computing Kernel for Network Binarization on PyTorch](https://mlsanity.com/paper/1911.04477/) (2019)

## Abstract (arXiv)

Binarized Neural Networks (BNNs) significantly reduce the computation and memory demands with binarized weights and activations compared to full-precision NNs. Executing a layer in a BNN on different devices of a heterogeneous multiprocessor platform consisting of CPU and GPU can affect the inference performance, i.e., accuracy and latency. Usually, a heterogeneous HW platform consisting of a CPU and a GPU is available to execute the BNN workloads. However, to use the heterogeneous HW effectively, it is necessary to find an efficient strategy for BNN workload mapping. In this work, we propose a framework that generates efficient BNN layer-to-device mappings (i.e. suitable parallel configuration for each layer of the model) for execution platforms comprised of CPU and CUDA-capable GPU. We evaluate our proposed framework with two BNN architectures using two well-known datasets, Fashion-MNIST and CIFAR-10, on three hardware platforms with different characteristics. The results show that compared to running a fully-parallelized GPU implementation, our framework generates an efficient configuration up to 2x, 2.6x and 11.8x faster on our tested hardware respectively.