# M4BRAM: Mixed-Precision Matrix-Matrix Multiplication in FPGA Block RAMs

arXiv: https://arxiv.org/abs/2311.02758 · HTML: https://mlsanity.com/paper/2311.02758/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 36 references that are in the map.
- First posted on arXiv 5 Nov 2023.

## Builds on (most cited first, 14 of 14)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [Ternary Weight Networks](https://mlsanity.com/paper/1605.04711/) (2016)
- [Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Networks](https://mlsanity.com/paper/1712.01507/) (2017)
- [AutoQ: Automated Kernel-Wise Neural Network Quantization](https://mlsanity.com/paper/1902.05690/) (2019)
- [HAWQV3: Dyadic Neural Network Quantization](https://mlsanity.com/paper/2011.10680/) (2020)
- [Adaptable Butterfly Accelerator for Attention-based NNs via Hardware and Algorithm Co-design](https://mlsanity.com/paper/2209.09570/) (2022)
- [F8Net: Fixed-Point 8-bit Only Multiplication for Network Quantization](https://mlsanity.com/paper/2202.05239/) (2022)
- [HAO: Hardware-aware neural Architecture Optimization for Efficient Inference](https://mlsanity.com/paper/2104.12766/) (2021)
- [DLA: Compiler and FPGA Overlay for Neural Network Inference Acceleration](https://mlsanity.com/paper/1807.06434/) (2018)
- [An OpenCL(TM) Deep Learning Accelerator on Arria 10](https://mlsanity.com/paper/1701.03534/) (2017)
- [Design Flow of Accelerating Hybrid Extremely Low Bit-width Neural Network in Embedded FPGA](https://mlsanity.com/paper/1808.04311/) (2018)
- [CoMeFa: Compute-in-Memory Blocks for FPGAs](https://mlsanity.com/paper/2203.12521/) (2022)
- [BRAMAC: Compute-in-BRAM Architectures for Multiply-Accumulate on FPGAs](https://mlsanity.com/paper/2304.03974/) (2023)

## Built on by (most cited first, 5 of 5)

- [Learning from Students: Applying t-Distributions to Explore Accurate and Efficient Formats for LLMs](https://mlsanity.com/paper/2405.03103/) (2024)
- [ITERA-LLM: Boosting Sub-8-Bit Large Language Model Inference via Iterative Tensor Decomposition](https://mlsanity.com/paper/2505.08981/) (2025)
- [Precision-Scalable Microscaling Datapaths with Optimized Reduction Tree for Efficient NPU Integration](https://mlsanity.com/paper/2511.06313/) (2025)
- [XtraMAC: An Efficient MAC Architecture for Mixed-Precision LLM Inference on FPGA](https://mlsanity.com/paper/2605.06052/) (2026)
- [NIFA: Nonlinear IMC enhanced FPGA for efficient ML inference](https://mlsanity.com/paper/2607.15123/) (2026)

## Abstract (arXiv)

Mixed-precision quantization is a popular approach for compressing deep neural networks (DNNs). However, it is challenging to scale the performance efficiently with mixed-precision DNNs given the current FPGA architecture and conventional accelerator dataflows. In this work, we enhance the FPGA's capability for accelerating mixed-precision DNNs by proposing M4BRAM, a novel compute-in-block RAM (BRAM) architecture that can compute mixed-precision matrix-matrix multiplication. On the precision side, M4BRAM supports a wide range of mixed-precision DNN configurations -- the weight precision can be 2/4/8 bits while the activation precision can vary from 2 to 8 bits. On the dataflow side, M4BRAM leverages a novel in-BRAM data duplication scheme to achieve high hardware utilization. Moreover, during M4BRAM computation, other FPGA resources can seamlessly access its data without the need for a separate buffer. Hence, unlike prior compute-in-BRAM proposals, M4BRAM can simultaneously perform mixed-precision computation and maintain full functionality as a memory unit to textittruly complement the existing compute resources on FPGAs. Experiments show that adding M4BRAM to a tiled DNN accelerator can achieve an average speedup of 2.16× across various DNNs on the ImageNet classification task while incurring a negligible accuracy loss of < 0.5%. Compared to the same tiled accelerator that employs a prior compute-in-BRAM architecture, M4BRAM delivers 1.43× higher performance on average across various DNNs.