# Synetgy: Algorithm-hardware Co-design for ConvNet Accelerators on Embedded FPGAs

arXiv: https://arxiv.org/abs/1811.08634 · HTML: https://mlsanity.com/paper/1811.08634/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Nov 2018.

## Built on by (most cited first, 18 of 18)

- [Dynamic Neural Networks: A Survey](https://mlsanity.com/paper/2102.04906/) (2021)
- [BiT: Robustly Binarized Multi-distilled Transformer](https://mlsanity.com/paper/2205.13016/) (2022)
- [How Do Adam and Training Strategies Help BNNs Optimization?](https://mlsanity.com/paper/2106.11309/) (2021)
- [HAO: Hardware-aware neural Architecture Optimization for Efficient Inference](https://mlsanity.com/paper/2104.12766/) (2021)
- [Rethinking Co-design of Neural Architectures and Hardware Accelerators](https://mlsanity.com/paper/2102.08619/) (2021)
- [LANA: Latency Aware Network Acceleration](https://mlsanity.com/paper/2107.10624/) (2021)
- [ViTA: A Vision Transformer Inference Accelerator for Edge Applications](https://mlsanity.com/paper/2302.09108/) (2023)
- [Tailor: Altering Skip Connections for Resource-Efficient Inference](https://mlsanity.com/paper/2301.07247/) (2023)
- [MetaML-Pro: Cross-Stage Design Flow Automation for Efficient Deep Learning Acceleration](https://mlsanity.com/paper/2502.05850/) (2025)
- [MetaML: Automating Customizable Cross-Stage Design-Flow for Deep Learning Acceleration](https://mlsanity.com/paper/2306.08746/) (2023)
- [Vis-TOP: Visual Transformer Overlay Processor](https://mlsanity.com/paper/2110.10957/) (2021)
- [Trainable Fixed-Point Quantization for Deep Learning Acceleration on FPGAs](https://mlsanity.com/paper/2401.17544/) (2024)
- [Torch2Chip: An End-to-end Customizable Deep Neural Network Compression and Deployment Toolkit for Prototype Hardware Accelerator Design](https://mlsanity.com/paper/2405.01775/) (2024)
- [Compilation and Optimizations for Efficient Machine Learning on Embedded Systems](https://mlsanity.com/paper/2206.03326/) (2022)
- [An FPGA-based Solution for Convolution Operation Acceleration](https://mlsanity.com/paper/2206.04520/) (2022)
- [Efficient Online Processing with Deep Neural Networks](https://mlsanity.com/paper/2306.13474/) (2023)
- [Efficiera Residual Networks: Hardware-Friendly Fully Binary Weight with 2-bit Activation Model Achieves Practical ImageNet Accuracy](https://mlsanity.com/paper/2410.11553/) (2024)
- [A Self-Ensemble Inspired Approach for Effective Training of Binary-Weight Spiking Neural Networks](https://mlsanity.com/paper/2508.12609/) (2025)

## Abstract (arXiv)

Using FPGAs to accelerate ConvNets has attracted significant attention in recent years. However, FPGA accelerator design has not leveraged the latest progress of ConvNets. As a result, the key application characteristics such as frames-per-second (FPS) are ignored in favor of simply counting GOPs, and results on accuracy, which is critical to application success, are often not even reported. In this work, we adopt an algorithm-hardware co-design approach to develop a ConvNet accelerator called Synetgy and a novel ConvNet model called DiracDeltaNetdagger. Both the accelerator and ConvNet are tailored to FPGA requirements. DiracDeltaNet, as the name suggests, is a ConvNet with only 1× 1 convolutions while spatial convolutions are replaced by more efficient shift operations. DiracDeltaNet achieves competitive accuracy on ImageNet (88.7% top-5), but with 42× fewer parameters and 48× fewer OPs than VGG16. We further quantize DiracDeltaNet's weights to 4-bit and activations to 4-bits, with less than 1% accuracy loss. These quantizations exploit well the nature of FPGA hardware. In short, DiracDeltaNet's small model size, low computational OP count, low precision and simplified operators allow us to co-design a highly customized computing unit for an FPGA. We implement the computing units for DiracDeltaNet on an Ultra96 SoC system through high-level synthesis. Our accelerator's final top-5 accuracy of 88.1% on ImageNet, is higher than all the previously reported embedded FPGA accelerators. In addition, the accelerator reaches an inference speed of 66.3 FPS on the ImageNet classification task, surpassing prior works with similar accuracy by at least 11.6×.