# Open-source FPGA-ML codesign for the MLPerf Tiny Benchmark

arXiv: https://arxiv.org/abs/2206.11791 · HTML: https://mlsanity.com/paper/2206.11791/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 58 references that are in the map.
- First posted on arXiv 23 Jun 2022.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [A System for Massively Parallel Hyperparameter Tuning](https://mlsanity.com/paper/1810.05934/) (2018)
- [FINN: A Framework for Fast, Scalable Binarized Neural Network Inference](https://mlsanity.com/paper/1612.07119/) (2016)
- [Fast inference of deep neural networks in FPGAs for particle physics](https://mlsanity.com/paper/1804.06913/) (2018)
- [MLPerf Tiny Benchmark](https://mlsanity.com/paper/2106.07597/) (2021)
- [Hello Edge: Keyword Spotting on Microcontrollers](https://mlsanity.com/paper/1711.07128/) (2017)
- [Benchmarking TinyML Systems: Challenges and Direction](https://mlsanity.com/paper/2003.04821/) (2020)
- [hls4ml: An Open-Source Codesign Workflow to Empower Scientific Low-Power Machine Learning Devices](https://mlsanity.com/paper/2103.05579/) (2021)
- [FINN-R: An End-to-End Deep-Learning Framework for Fast Exploration of Quantized Neural Networks](https://mlsanity.com/paper/1809.04570/) (2018)
- [ToyADMOS: A Dataset of Miniature-Machine Operating Sounds for Anomalous Sound Detection](https://mlsanity.com/paper/1908.03299/) (2019)
- [Compressing deep neural networks on FPGAs to binary and ternary precision with HLS4ML](https://mlsanity.com/paper/2003.06308/) (2020)
- [Fast convolutional neural networks on FPGAs with hls4ml](https://mlsanity.com/paper/2101.05108/) (2021)
- [Distance-Weighted Graph Neural Networks on FPGAs for Real-Time Particle Reconstruction in High Energy Physics](https://mlsanity.com/paper/2008.03601/) (2020)
- [Applications and Techniques for Fast Machine Learning in Science](https://mlsanity.com/paper/2110.13041/) (2021)
- [Ps and Qs: Quantization-aware pruning for efficient low latency neural network inference](https://mlsanity.com/paper/2102.11289/) (2021)
- [FPGA-based Accelerators of Deep Learning Networks for Learning and Classification: A Review](https://mlsanity.com/paper/1901.00121/) (2019)
- [Toolflows for Mapping Convolutional Neural Networks on FPGAs: A Survey and Future Directions](https://mlsanity.com/paper/1803.05900/) (2018)
- [Mix and Match: A Novel FPGA-Centric Deep Neural Network Quantization Framework](https://mlsanity.com/paper/2012.04240/) (2020)
- [Fast inference of Boosted Decision Trees in FPGAs for particle physics](https://mlsanity.com/paper/2002.02534/) (2020)
- [Graph Neural Networks for Charged Particle Tracking on FPGAs](https://mlsanity.com/paper/2112.02048/) (2021)
- [Accelerating CNN inference on FPGAs: A Survey](https://mlsanity.com/paper/1806.01683/) (2018)
- [Streamlined Deployment for Quantized Neural Networks](https://mlsanity.com/paper/1709.04060/) (2017)
- [QONNX: Representing Arbitrary-Precision Quantized Neural Networks](https://mlsanity.com/paper/2206.07527/) (2022)
- [FixyNN: Efficient Hardware for Mobile Computer Vision via Transfer Learning](https://mlsanity.com/paper/1902.11128/) (2019)
- [Caffeinated FPGAs: FPGA Framework For Convolutional Neural Networks](https://mlsanity.com/paper/1609.09671/) (2016)
- [fpgaConvNet: A Toolflow for Mapping Diverse Convolutional Neural Networks on Embedded FPGAs](https://mlsanity.com/paper/1711.08740/) (2017)
- [Quantized Guided Pruning for Efficient Hardware Implementations of Convolutional Neural Networks](https://mlsanity.com/paper/1812.11337/) (2018)

## Built on by (most cited first, 9 of 9)

- [hls4ml: A Flexible, Open-Source Platform for Deep Learning Acceleration on Reconfigurable Hardware](https://mlsanity.com/paper/2512.01463/) (2025)
- [Tailor: Altering Skip Connections for Resource-Efficient Inference](https://mlsanity.com/paper/2301.07247/) (2023)
- [rule4ml: An Open-Source Tool for Resource Utilization and Latency Estimation for ML Models on FPGA](https://mlsanity.com/paper/2408.05314/) (2024)
- [wa-hls4ml: A Benchmark and Surrogate Models for hls4ml Resource and Latency Estimation](https://mlsanity.com/paper/2511.05615/) (2025)
- [Architectural Implications of Neural Network Inference for High Data-Rate, Low-Latency Scientific Applications](https://mlsanity.com/paper/2403.08980/) (2024)
- [CGRA4ML: A Hardware/Software Framework to Implement Neural Networks for Scientific Edge Computing](https://mlsanity.com/paper/2408.15561/) (2024)
- [Neural network accelerator for quantum control](https://mlsanity.com/paper/2208.02645/) (2022)
- [PEFSL: A deployment Pipeline for Embedded Few-Shot Learning on a FPGA SoC](https://mlsanity.com/paper/2404.19354/) (2024)
- [BitLogic: Training Framework for Gradient-Based FPGA-Native Neural Networks](https://mlsanity.com/paper/2602.07400/) (2026)

## Abstract (arXiv)

We present our development experience and recent results for the MLPerf Tiny Inference Benchmark on field-programmable gate array (FPGA) platforms. We use the open-source hls4ml and FINN workflows, which aim to democratize AI-hardware codesign of optimized neural networks on FPGAs. We present the design and implementation process for the keyword spotting, anomaly detection, and image classification benchmark tasks. The resulting hardware implementations are quantized, configurable, spatial dataflow architectures tailored for speed and efficiency and introduce new generic optimizations and common workflows developed as a part of this work. The full workflow is presented from quantization-aware training to FPGA implementation. The solutions are deployed on system-on-chip (Pynq-Z2) and pure FPGA (Arty A7-100T) platforms. The resulting submissions achieve latencies as low as 20 μs and energy consumption as low as 30 μJ per inference. We demonstrate how emerging ML benchmarks on heterogeneous hardware platforms can catalyze collaboration and the development of new techniques and more accessible tools.