# FPGA Resource-aware Structured Pruning for Real-Time Neural Networks

arXiv: https://arxiv.org/abs/2308.05170 · HTML: https://mlsanity.com/paper/2308.05170/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 46 references that are in the map.
- First posted on arXiv 9 Aug 2023.

## Builds on (most cited first, 28 of 28)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Pruning Filters for Efficient ConvNets](https://mlsanity.com/paper/1608.08710/) (2016)
- [Learning Efficient Convolutional Networks through Network Slimming](https://mlsanity.com/paper/1708.06519/) (2017)
- [What is the State of Neural Network Pruning?](https://mlsanity.com/paper/2003.03033/) (2020)
- [Learning Structured Sparsity in Deep Neural Networks](https://mlsanity.com/paper/1608.03665/) (2016)
- [Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks](https://mlsanity.com/paper/2102.00554/) (2021)
- [FINN: A Framework for Fast, Scalable Binarized Neural Network Inference](https://mlsanity.com/paper/1612.07119/) (2016)
- [Fast inference of deep neural networks in FPGAs for particle physics](https://mlsanity.com/paper/1804.06913/) (2018)
- [Bayesian Bits: Unifying Quantization and Pruning](https://mlsanity.com/paper/2005.07093/) (2020)
- [Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors](https://mlsanity.com/paper/2006.10159/) (2020)
- [PatDNN: Achieving Real-Time DNN Execution on Mobile Devices with Pattern-based Weight Pruning](https://mlsanity.com/paper/2001.00138/) (2020)
- [FINN-R: An End-to-End Deep-Learning Framework for Fast Exploration of Quantized Neural Networks](https://mlsanity.com/paper/1809.04570/) (2018)
- [Methods for Pruning Deep Neural Networks](https://mlsanity.com/paper/2011.00241/) (2020)
- [ADMM-NN: An Algorithm-Hardware Co-Design Framework of DNNs Using Alternating Direction Method of Multipliers](https://mlsanity.com/paper/1812.11677/) (2018)
- [Compressing deep neural networks on FPGAs to binary and ternary precision with HLS4ML](https://mlsanity.com/paper/2003.06308/) (2020)
- [Fast convolutional neural networks on FPGAs with hls4ml](https://mlsanity.com/paper/2101.05108/) (2021)
- [Structural Pruning via Latency-Saliency Knapsack](https://mlsanity.com/paper/2210.06659/) (2022)
- [Distance-Weighted Graph Neural Networks on FPGAs for Real-Time Particle Reconstruction in High Energy Physics](https://mlsanity.com/paper/2008.03601/) (2020)
- [LUTNet: Rethinking Inference in FPGA Soft Logic](https://mlsanity.com/paper/1904.00938/) (2019)
- [Accelerator-Aware Pruning for Convolutional Neural Networks](https://mlsanity.com/paper/1804.09862/) (2018)
- [Ultra-low latency recurrent neural network inference on FPGAs for physics applications with hls4ml](https://mlsanity.com/paper/2207.00559/) (2022)
- [Real-time semantic segmentation on FPGAs for autonomous vehicles with hls4ml](https://mlsanity.com/paper/2205.07690/) (2022)
- [PCNN: Pattern-based Fine-Grained Regular Pruning towards Optimizing CNN Accelerators](https://mlsanity.com/paper/2002.04997/) (2020)
- [Joint Pruning & Quantization for Extremely Sparse Neural Networks](https://mlsanity.com/paper/2010.01892/) (2020)
- [HALP: Hardware-Aware Latency Pruning](https://mlsanity.com/paper/2110.10811/) (2021)
- [Logic Shrinkage: Learned FPGA Netlist Sparsity for Efficient Neural Network Inference](https://mlsanity.com/paper/2112.02346/) (2021)

## Built on by (most cited first, 7 of 7)

- [HGQ: High Granularity Quantization for Real-time Neural Networks on FPGAs](https://mlsanity.com/paper/2405.00645/) (2024)
- [hls4ml: A Flexible, Open-Source Platform for Deep Learning Acceleration on Reconfigurable Hardware](https://mlsanity.com/paper/2512.01463/) (2025)
- [da4ml: Distributed Arithmetic for Real-time Neural Networks on FPGAs](https://mlsanity.com/paper/2507.04535/) (2025)
- [Optimal Gradient Checkpointing for Sparse and Recurrent Architectures using Off-Chip Memory](https://mlsanity.com/paper/2412.11810/) (2024)
- [HGQ-LUT: Fast LUT-Aware Training and Efficient Architectures for DNN Inference](https://mlsanity.com/paper/2604.22293/) (2026)
- [PQuantML: A Tool for End-to-End Hardware-aware Model Compression](https://mlsanity.com/paper/2603.26595/) (2026)
- [Hardware-aware Graph Neural Networks prunning for embedded event-based vision](https://mlsanity.com/paper/2607.06739/) (2026)

## Abstract (arXiv)

Neural networks achieve state-of-the-art performance in image classification, speech recognition, scientific analysis and many more application areas. Due to the high computational complexity and memory footprint of neural networks, various compression techniques, such as pruning and quantization, have been proposed in literature. Pruning sparsifies a neural network, reducing the number of multiplications and memory. However, pruning often fails to capture properties of the underlying hardware, causing unstructured sparsity and load-balance inefficiency, thus bottlenecking resource improvements. We propose a hardware-centric formulation of pruning, by formulating it as a knapsack problem with resource-aware tensor structures. Evaluated on a range of tasks, including sub-microsecond particle classification at CERN's Large Hadron Collider and fast image classification, the proposed method achieves reductions ranging between 55% and 92% in the DSP utilization and up to 81% in BRAM utilization.