# Efficient Edge AI: Deploying Convolutional Neural Networks on FPGA with the Gemmini Accelerator

arXiv: https://arxiv.org/abs/2408.07404 · HTML: https://mlsanity.com/paper/2408.07404/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 38 references that are in the map.
- First posted on arXiv 14 Aug 2024.

## Builds on (most cited first, 13 of 13)

- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors](https://mlsanity.com/paper/2207.02696/) (2022)
- [Robot Operating System 2: Design, Architecture, and Uses In The Wild](https://mlsanity.com/paper/2211.07752/) (2022)
- [Learning to Optimize Tensor Programs](https://mlsanity.com/paper/1805.08166/) (2018)
- [Gemmini: Enabling Systematic Deep-Learning Architecture Evaluation via Full-Stack Integration](https://mlsanity.com/paper/1911.09925/) (2019)
- [FINN-R: An End-to-End Deep-Learning Framework for Fast Exploration of Quantized Neural Networks](https://mlsanity.com/paper/1809.04570/) (2018)
- [Fast convolutional neural networks on FPGAs with hls4ml](https://mlsanity.com/paper/2101.05108/) (2021)
- [Toolflows for Mapping Convolutional Neural Networks on FPGAs: A Survey and Future Directions](https://mlsanity.com/paper/1803.05900/) (2018)
- [A Hardware-Software Blueprint for Flexible Deep Learning Specialization](https://mlsanity.com/paper/1807.04188/) (2018)
- [DSP-Packing: Squeezing Low-precision Arithmetic into FPGA DSP Blocks](https://mlsanity.com/paper/2203.11028/) (2022)
- [Sparse Winograd Convolutional neural networks on small-scale systolic arrays](https://mlsanity.com/paper/1810.01973/) (2018)
- [Iterative Filter Pruning for Concatenation-based CNN Architectures](https://mlsanity.com/paper/2405.03715/) (2024)
- [Integration of a systolic array based hardware accelerator into a DNN operator auto-tuning framework](https://mlsanity.com/paper/2212.03034/) (2022)

## Built on by (most cited first, 1 of 1)

- [On-Device Adaptive Battery Power Prediction for Electric Vehicles](https://mlsanity.com/paper/2607.09400/) (2026)

## Abstract (arXiv)

The growing concerns regarding energy consumption and privacy have prompted the development of AI solutions deployable on the edge, circumventing the substantial CO2 emissions associated with cloud servers and mitigating risks related to sharing sensitive data. But deploying Convolutional Neural Networks (CNNs) on non-off-the-shelf edge devices remains a complex and labor-intensive task. In this paper, we present and end-to-end workflow for deployment of CNNs on Field Programmable Gate Arrays (FPGAs) using the Gemmini accelerator, which we modified for efficient implementation on FPGAs. We describe how we leverage the use of open source software on each optimization step of the deployment process, the customizations we added to them and its impact on the final system's performance. We were able to achieve real-time performance by deploying a YOLOv7 model on a Xilinx ZCU102 FPGA with an energy efficiency of 36.5 GOP/s/W. Our FPGA-based solution demonstrates superior power efficiency compared with other embedded hardware devices, and even outperforms other FPGA reference implementations. Finally, we present how this kind of solution can be integrated into a wider system, by testing our proposed platform in a traffic monitoring scenario.