# Mitigating scalability challenges in LUT-based neural networks via pruning optimisations

arXiv: https://arxiv.org/abs/2407.02362 · HTML: https://mlsanity.com/paper/2407.02362/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 34 references that are in the map.
- First posted on arXiv 2 Jul 2024.

## Builds on (most cited first, 13 of 13)

- [FINN: A Framework for Fast, Scalable Binarized Neural Network Inference](https://mlsanity.com/paper/1612.07119/) (2016)
- [FracBNN: Accurate and FPGA-Efficient Binary Neural Networks with Fractional Activations](https://mlsanity.com/paper/2012.12206/) (2020)
- [Multiplying Matrices Without Multiplying](https://mlsanity.com/paper/2106.10860/) (2021)
- [Quantization based Fast Inner Product Search](https://mlsanity.com/paper/1509.01469/) (2015)
- [LUTNet: Learning FPGA Configurations for Highly Efficient Neural Network Inference](https://mlsanity.com/paper/1910.12625/) (2019)
- [LUT-NN: Empower Efficient Neural Network Inference with Centroid Learning and Table Lookup](https://mlsanity.com/paper/2302.03213/) (2023)
- [Streamlined Deployment for Quantized Neural Networks](https://mlsanity.com/paper/1709.04060/) (2017)
- [Resource-Efficient Deep Learning: A Survey on Model-, Arithmetic-, and Implementation-Level Techniques](https://mlsanity.com/paper/2112.15131/) (2021)
- [Accelerating Sparse DNNs Based on Tiled GEMM](https://mlsanity.com/paper/2402.10876/) (2024)
- [PQA: Exploring the Potential of Product Quantization in DNN Hardware Acceleration](https://mlsanity.com/paper/2305.18334/) (2023)
- [Sparse-DySta: Sparsity-Aware Dynamic and Static Scheduling for Sparse Multi-DNN Workloads](https://mlsanity.com/paper/2310.11096/) (2023)
- [Table-Lookup MAC: Scalable Processing of Quantised Neural Networks in FPGA Soft Logic](https://mlsanity.com/paper/2403.11414/) (2024)
- [TabConv: Low-Computation CNN Inference via Table Lookups](https://mlsanity.com/paper/2404.05872/) (2024)

## Abstract (arXiv)

Modern deep neural networks heavily rely on a large number of multiply-accumulate operations, which constitute the predominant computational cost. To address this, Look-Up Table (LUT)-based matrix multiplications have emerged as a promising alternative for reducing the computational cost and time of the multiply-accumulate operations in a neural network. However, the LUT-based neural network still faces the scalability challenge due to the inherent limitations of LUT-based matrix multiplication. To mitigate these scalability limitations, this paper proposes a scalable and energy-efficient LUT-based approximate matrix multiplication unit (LUT-MU) constituting the basic component of the neural networks by integrating a pruning strategy on the MADDNESS algorithm, a LUT-based matrix multiplication methodology. With increasing problem size and precision demands in matrix multiplication, our proposed LUT-MU architecture effectively constrains resource expansion. The case study shows that deploying our LUT-MU in neural network architectures, including fully connected layers (MNIST) and ResNets (CIFAR-10, ImageNet)-on XCZU7EV and XCZU19EG FPGAs, produces up to 1.6 × throughput improvement and 4.2 × energy efficiency gains over mainstream CUDA-based network implementations, and 1.8× energy efficiency compared to leading quantised neural network implementations, with moderate impact on accuracy. Compared to original MADDNESS-based neural networks, our LUT-MU shows 1.3 to 2.6× resource savings based on various resolution configuration settings of MADDNESS.