# Mix and Match: A Novel FPGA-Centric Deep Neural Network Quantization Framework

arXiv: https://arxiv.org/abs/2012.04240 · HTML: https://mlsanity.com/paper/2012.04240/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Dec 2020.

## Built on by (most cited first, 27 of 27)

- [SPViT: Enabling Faster Vision Transformers via Soft Token Pruning](https://mlsanity.com/paper/2112.13890/) (2021)
- [hls4ml: An Open-Source Codesign Workflow to Empower Scientific Low-Power Machine Learning Devices](https://mlsanity.com/paper/2103.05579/) (2021)
- [A survey on efficient vision transformers: algorithms, techniques, and performance benchmarking](https://mlsanity.com/paper/2309.02031/) (2023)
- [Ps and Qs: Quantization-aware pruning for efficient low latency neural network inference](https://mlsanity.com/paper/2102.11289/) (2021)
- [Auto-ViT-Acc: An FPGA-Aware Automatic Acceleration Framework for Vision Transformer with Mixed-Scheme Quantization](https://mlsanity.com/paper/2208.05163/) (2022)
- [Ditto: Accelerating Diffusion Model via Temporal Value Similarity](https://mlsanity.com/paper/2501.11211/) (2025)
- [Neural Network Quantization for Efficient Inference: A Survey](https://mlsanity.com/paper/2112.06126/) (2021)
- [Oaken: Fast and Efficient LLM Serving with Online-Offline Hybrid KV Cache Quantization](https://mlsanity.com/paper/2503.18599/) (2025)
- [Quantum Neural Network Compression](https://mlsanity.com/paper/2207.01578/) (2022)
- [A Precision-Scalable RISC-V DNN Processor with On-Device Learning Capability at the Extreme Edge](https://mlsanity.com/paper/2309.08186/) (2023)
- [Open-source FPGA-ML codesign for the MLPerf Tiny Benchmark](https://mlsanity.com/paper/2206.11791/) (2022)
- [HGQ: High Granularity Quantization for Real-time Neural Networks on FPGAs](https://mlsanity.com/paper/2405.00645/) (2024)
- [RMSMP: A Novel Deep Neural Network Quantization Framework with Row-wise Mixed Schemes and Multiple Precisions](https://mlsanity.com/paper/2111.00153/) (2021)
- [Tailor: Altering Skip Connections for Resource-Efficient Inference](https://mlsanity.com/paper/2301.07247/) (2023)
- [BOMP-NAS: Bayesian Optimization Mixed Precision NAS](https://mlsanity.com/paper/2301.11810/) (2023)
- [Bespoke Approximation of Multiplication-Accumulation and Activation Targeting Printed Multilayer Perceptrons](https://mlsanity.com/paper/2312.17612/) (2023)
- [N3H-Core: Neuron-designed Neural Network Accelerator via FPGA-based Heterogeneous Computing Cores](https://mlsanity.com/paper/2112.08193/) (2021)
- [Real-time Hyper-Dimensional Reconfiguration at the Edge using Hardware Accelerators](https://mlsanity.com/paper/2206.05128/) (2022)
- [LUTMUL: Exceed Conventional FPGA Roofline Limit by LUT-based Efficient Multiplication for Neural Network Inference](https://mlsanity.com/paper/2411.11852/) (2024)
- [Accelerating PoT Quantization on Edge Devices](https://mlsanity.com/paper/2409.20403/) (2024)
- [SwiftChannel: Algorithm-Hardware Co-Design for Deep Learning-Based 5G Channel Estimation](https://mlsanity.com/paper/2605.01931/) (2026)
- [PoTAcc: A Pipeline for End-to-End Acceleration of Power-of-Two Quantized DNNs](https://mlsanity.com/paper/2605.06082/) (2026)
- [A Deep Learning Inference Scheme Based on Pipelined Matrix Multiplication Acceleration Design and Non-uniform Quantization](https://mlsanity.com/paper/2110.04861/) (2021)
- [ILMPQ : An Intra-Layer Multi-Precision Deep Neural Network Quantization framework for FPGA](https://mlsanity.com/paper/2111.00155/) (2021)
- [Real-Time Portrait Stylization on the Edge](https://mlsanity.com/paper/2206.01244/) (2022)
- [FullPack: Full Vector Utilization for Sub-Byte Quantized Inference on General Purpose CPUs](https://mlsanity.com/paper/2211.06982/) (2022)
- [ViM-Q: Scalable Algorithm-Hardware Co-Design for Vision Mamba Model Inference on FPGA](https://mlsanity.com/paper/2605.01935/) (2026)

## Abstract (arXiv)

Deep Neural Networks (DNNs) have achieved extraordinary performance in various application domains. To support diverse DNN models, efficient implementations of DNN inference on edge-computing platforms, e.g., ASICs, FPGAs, and embedded systems, are extensively investigated. Due to the huge model size and computation amount, model compression is a critical step to deploy DNN models on edge devices. This paper focuses on weight quantization, a hardware-friendly model compression approach that is complementary to weight pruning. Unlike existing methods that use the same quantization scheme for all weights, we propose the first solution that applies different quantization schemes for different rows of the weight matrix. It is motivated by (1) the distribution of the weights in the different rows are not the same; and (2) the potential of achieving better utilization of heterogeneous FPGA hardware resources. To achieve that, we first propose a hardware-friendly quantization scheme named sum-of-power-of-2 (SP2) suitable for Gaussian-like weight distribution, in which the multiplication arithmetic can be replaced with logic shifter and adder, thereby enabling highly efficient implementations with the FPGA LUT resources. In contrast, the existing fixed-point quantization is suitable for Uniform-like weight distribution and can be implemented efficiently by DSP. Then to fully explore the resources, we propose an FPGA-centric mixed scheme quantization (MSQ) with an ensemble of the proposed SP2 and the fixed-point schemes. Combining the two schemes can maintain, or even increase accuracy due to better matching with weight distributions.