# PolyLUT: Ultra-low Latency Polynomial Inference with Hardware-Aware Structured Pruning

arXiv: https://arxiv.org/abs/2501.08043 · HTML: https://mlsanity.com/paper/2501.08043/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 25 references that are in the map.
- First posted on arXiv 14 Jan 2025.

## Builds on (most cited first, 14 of 14)

- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [FINN: A Framework for Fast, Scalable Binarized Neural Network Inference](https://mlsanity.com/paper/1612.07119/) (2016)
- [Fast inference of deep neural networks in FPGAs for particle physics](https://mlsanity.com/paper/1804.06913/) (2018)
- [Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors](https://mlsanity.com/paper/2006.10159/) (2020)
- [hls4ml: An Open-Source Codesign Workflow to Empower Scientific Low-Power Machine Learning Devices](https://mlsanity.com/paper/2103.05579/) (2021)
- [LogicNets: Co-Designed Neural Networks and Circuits for Extreme-Throughput Applications](https://mlsanity.com/paper/2004.03021/) (2020)
- [Compressing deep neural networks on FPGAs to binary and ternary precision with HLS4ML](https://mlsanity.com/paper/2003.06308/) (2020)
- [PolyLUT: Learning Piecewise Polynomials for Ultra-Low Latency FPGA LUT-based Inference](https://mlsanity.com/paper/2309.02334/) (2023)
- [LUTNet: Rethinking Inference in FPGA Soft Logic](https://mlsanity.com/paper/1904.00938/) (2019)
- [Fast inference of Boosted Decision Trees in FPGAs for particle physics](https://mlsanity.com/paper/2002.02534/) (2020)
- [LUTNet: Learning FPGA Configurations for Highly Efficient Neural Network Inference](https://mlsanity.com/paper/1910.12625/) (2019)
- [Deep Neural Network Approximation for Custom Hardware: Where We've Been, Where We're Going](https://mlsanity.com/paper/1901.06955/) (2019)

## Built on by (most cited first, 8 of 8)

- [NeuraLUT-Assemble: Hardware-aware Assembling of Sub-Neural Networks for Efficient LUT Inference](https://mlsanity.com/paper/2504.00592/) (2025)
- [Light Differentiable Logic Gate Networks](https://mlsanity.com/paper/2510.03250/) (2025)
- [hls4ml: A Flexible, Open-Source Platform for Deep Learning Acceleration on Reconfigurable Hardware](https://mlsanity.com/paper/2512.01463/) (2025)
- [da4ml: Distributed Arithmetic for Real-time Neural Networks on FPGAs](https://mlsanity.com/paper/2507.04535/) (2025)
- [BitLogic: Training Framework for Gradient-Based FPGA-Native Neural Networks](https://mlsanity.com/paper/2602.07400/) (2026)
- [HGQ-LUT: Fast LUT-Aware Training and Efficient Architectures for DNN Inference](https://mlsanity.com/paper/2604.22293/) (2026)
- [Enhancing LUT-based Deep Neural Networks Inference through Architecture and Connectivity Optimization](https://mlsanity.com/paper/2601.09773/) (2026)
- [CascadeLUT: Information-Ordered Streaming Inference for Bandwidth-Constrained FPGAs](https://mlsanity.com/paper/2608.00720/) (2026)

## Abstract (arXiv)

Standard deep neural network inference involves the computation of interleaved linear maps and nonlinear activation functions. Prior work for ultra-low latency implementations has hardcoded these operations inside FPGA lookup tables (LUTs). However, FPGA LUTs can implement a much greater variety of functions. In this paper, we propose a novel approach to training DNNs for FPGA deployment using multivariate polynomials as the basic building block. Our method takes advantage of the flexibility offered by the soft logic, hiding the polynomial evaluation inside the LUTs with minimal overhead. By using polynomial building blocks, we achieve the same accuracy using considerably fewer layers of soft logic than by using linear functions, leading to significant latency and area improvements. LUT-based implementations also face a significant challenge: the LUT size grows exponentially with the number of inputs. Prior work relies on a priori fixed sparsity, with results heavily dependent on seed selection. To address this, we propose a structured pruning strategy using a bespoke hardware-aware group regularizer that encourages a particular sparsity pattern that leads to a small number of inputs per neuron. We demonstrate the effectiveness of PolyLUT on three tasks: network intrusion detection, jet identification at the CERN Large Hadron Collider, and MNIST.