# A Compilation Flow for the Generation of CNN Inference Accelerators on FPGAs

arXiv: https://arxiv.org/abs/2203.04015 · HTML: https://mlsanity.com/paper/2203.04015/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 26 references that are in the map.
- First posted on arXiv 8 Mar 2022.

## Builds on (most cited first, 7 of 7)

- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Toolflows for Mapping Convolutional Neural Networks on FPGAs: A Survey and Future Directions](https://mlsanity.com/paper/1803.05900/) (2018)
- [Relay: A New IR for Machine Learning Frameworks](https://mlsanity.com/paper/1810.00952/) (2018)
- [DLA: Compiler and FPGA Overlay for Neural Network Inference Acceleration](https://mlsanity.com/paper/1807.06434/) (2018)
- [Caffeinated FPGAs: FPGA Framework For Convolutional Neural Networks](https://mlsanity.com/paper/1609.09671/) (2016)
- [HPIPE: Heterogeneous Layer-Pipelined and Sparse-Aware CNN Inference for FPGAs](https://mlsanity.com/paper/2007.10451/) (2020)
- [High Performance Computing with FPGAs and OpenCL](https://mlsanity.com/paper/1810.09773/) (2018)

## Abstract (arXiv)

We present a compilation flow for the generation of CNN inference accelerators on FPGAs. The flow translates a frozen model into OpenCL kernels with the TVM compiler and uses the Intel OpenCL SDK to compile to an FPGA bitstream. We improve the quality of the generated hardware with optimizations applied to the base OpenCL kernels generated by TVM. These optimizations increase parallelism, reduce memory access latency, increase concurrency and save on-chip resources. We automate these optimizations in TVM and evaluate them by generating accelerators for LeNet-5, MobileNetV1 and ResNet-34 on an Intel Stratix~10SX. We show that the optimizations improve the performance of the generated accelerators by up to 846X over the base accelerators. The performance of the optimized accelerators is up to 4.57X better than TensorFlow on CPU, 3.83X better than single-threaded TVM and is only 0.34X compared to TVM with 56 threads. Our optimized kernels also outperform ones generated by a similar approach (that also uses high-level synthesis) while providing more functionality and flexibility. However, it underperforms an approach that utilizes hand-optimized designs. Thus, we view our approach as useful in pre-production environments that benefit from increased performance and fast prototyping, realizing the benefits of FPGAs without hardware design expertise.