# DLA: Compiler and FPGA Overlay for Neural Network Inference Acceleration

arXiv: https://arxiv.org/abs/1807.06434 · HTML: https://mlsanity.com/paper/1807.06434/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jul 2018.

## Built on by (most cited first, 15 of 15)

- [AI Accelerator Survey and Trends](https://mlsanity.com/paper/2109.08957/) (2021)
- [PQA: Exploring the Potential of Product Quantization in DNN Hardware Acceleration](https://mlsanity.com/paper/2305.18334/) (2023)
- [FLIQS: One-Shot Mixed-Precision Floating-Point and Integer Quantization Search](https://mlsanity.com/paper/2308.03290/) (2023)
- [M4BRAM: Mixed-Precision Matrix-Matrix Multiplication in FPGA Block RAMs](https://mlsanity.com/paper/2311.02758/) (2023)
- [unzipFPGA: Enhancing FPGA-based CNN Engines with On-the-Fly Weights Generation](https://mlsanity.com/paper/2103.05600/) (2021)
- [BRAMAC: Compute-in-BRAM Architectures for Multiply-Accumulate on FPGAs](https://mlsanity.com/paper/2304.03974/) (2023)
- [H2PIPE: High throughput CNN Inference on FPGAs with High-Bandwidth Memory](https://mlsanity.com/paper/2408.09209/) (2024)
- [Vis-TOP: Visual Transformer Overlay Processor](https://mlsanity.com/paper/2110.10957/) (2021)
- [Lincoln AI Computing Survey (LAICS) Update](https://mlsanity.com/paper/2310.09145/) (2023)
- [GCV-Turbo: End-to-end Acceleration of GNN-based Computer Vision Tasks on FPGA](https://mlsanity.com/paper/2404.07188/) (2024)
- [FPGA-Enabled Machine Learning Applications in Earth Observation: A Systematic Review](https://mlsanity.com/paper/2506.03938/) (2025)
- [Mitigating Memory Wall Effects in CNN Engines with On-the-Fly Weights Generation](https://mlsanity.com/paper/2307.13412/) (2023)
- [Beyond Inference: Performance Analysis of DNN Server Overheads for Computer Vision](https://mlsanity.com/paper/2403.12981/) (2024)
- [A Compilation Flow for the Generation of CNN Inference Accelerators on FPGAs](https://mlsanity.com/paper/2203.04015/) (2022)
- [DiviML: A Module-based Heuristic for Mapping Neural Networks onto Heterogeneous Platforms](https://mlsanity.com/paper/2308.00127/) (2023)

## Abstract (arXiv)

Overlays have shown significant promise for field-programmable gate-arrays (FPGAs) as they allow for fast development cycles and remove many of the challenges of the traditional FPGA hardware design flow. However, this often comes with a significant performance burden resulting in very little adoption of overlays for practical applications. In this paper, we tailor an overlay to a specific application domain, and we show how we maintain its full programmability without paying for the performance overhead traditionally associated with overlays. Specifically, we introduce an overlay targeted for deep neural network inference with only ~1% overhead to support the control and reprogramming logic using a lightweight very-long instruction word (VLIW) network. Additionally, we implement a sophisticated domain specific graph compiler that compiles deep learning languages such as Caffe or Tensorflow to easily target our overlay. We show how our graph compiler performs architecture-driven software optimizations to significantly boost performance of both convolutional and recurrent neural networks (CNNs/RNNs) - we demonstrate a 3x improvement on ResNet-101 and a 12x improvement for long short-term memory (LSTM) cells, compared to naive implementations. Finally, we describe how we can tailor our hardware overlay, and use our graph compiler to achieve ~900 fps on GoogLeNet on an Intel Arria 10 1150 - the fastest ever reported on comparable FPGAs.