# Synergy: A HW/SW Framework for High Throughput CNNs on Embedded Heterogeneous SoC

arXiv: https://arxiv.org/abs/1804.00706 · HTML: https://mlsanity.com/paper/1804.00706/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Mar 2018.

## Built on by (most cited first, 4 of 4)

- [AutoFL: Enabling Heterogeneity-Aware Energy Efficient Federated Learning](https://mlsanity.com/paper/2107.08147/) (2021)
- [A Survey of Large-Scale Deep Learning Serving System Optimization: Challenges and Opportunities](https://mlsanity.com/paper/2111.14247/) (2021)
- [Video Individual Counting and Tracking from Moving Drones: A Benchmark and Methods](https://mlsanity.com/paper/2601.12500/) (2026)
- [SurgWound-Bench: A Benchmark for Surgical Wound Diagnosis](https://mlsanity.com/paper/2508.15189/) (2025)

## Abstract (arXiv)

Convolutional Neural Networks (CNN) have been widely deployed in diverse application domains. There has been significant progress in accelerating both their training and inference using high-performance GPUs, FPGAs, and custom ASICs for datacenter-scale environments. The recent proliferation of mobile and IoT devices have necessitated real-time, energy-efficient deep neural network inference on embedded-class, resource-constrained platforms. In this context, we present em Synergy, an automated, hardware-software co-designed, pipelined, high-throughput CNN inference framework on embedded heterogeneous system-on-chip (SoC) architectures (Xilinx Zynq). em Synergy leverages, through multi-threading, all the available on-chip resources, which includes the dual-core ARM processor along with the FPGA and the NEON SIMD engines as accelerators. Moreover, em Synergy provides a unified abstraction of the heterogeneous accelerators (FPGA and NEON) and can adapt to different network configurations at runtime without changing the underlying hardware accelerator architecture by balancing workload across accelerators through work-stealing. em Synergy achieves 7.3X speedup, averaged across seven CNN models, over a well-optimized software-only solution. em Synergy demonstrates substantially better throughput and energy-efficiency compared to the contemporary CNN implementations on the same SoC architecture.