mlsanityData to 14 Oct 2026

XtraMAC: An Efficient MAC Architecture for Mixed-Precision LLM Inference on FPGA

arXiv 2605.06052 ↗First posted 7 May 2026cs.ARcited by 0

Not on the map. Why?

Abstract (arXiv)

The widespread adoption of mixed-precision quantization in large language models (LLMs) has created demand for hardware that can efficiently perform multiply-accumulate (MAC) operations across mixed datatypes and switch datatypes at runtime. Existing FPGA-based MAC solutions fall short due to limitations in fixed-datatype design, inefficient spatial or temporal resource sharing, and poor support for mixed-precision execution. These limitations collectively lead to under-utilization of DSP resources, limiting achievable parallelism and throughput. In this work, we present XtraMAC, a novel MAC architecture that unifies integer, floating-point, and mixed-precision operations within a single, datatype-adaptive microarchitecture. XtraMAC decomposes all supported MAC formats into a shared integer mantissa product with lightweight sign and exponent handling, enabling dynamic operand packing and efficient DSP resource sharing with constant latency and initiation interval of one across all datatypes. Evaluated on an AMD Xilinx U55c FPGA, XtraMAC achieves 1.4-2.0x higher compute density, reduces per-operation LUT, FF, and DSP consumption by 27-51%, and delivers up to 1.9x greater energy efficiency and 1.2x speedup on representative mixed-precision LLM workloads. The implementation of XtraMAC is open-sourced at https://github.com/Xtra-Computing/XtraMAC.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 41 references that are in the map. First posted on arXiv 7 May 2026.

Builds on

19 of 41 references.

References · 19

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  4. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  5. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,386
  6. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  7. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 618
  8. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 414
  9. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 367
  10. 2023Atom: Low-bit Quantization for Efficient and Accurate LLM Servingcited by 272
  11. 2016FINN: A Framework for Fast, Scalable Binarized Neural Network Inferencecited by 150
  12. 2017Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Networkscited by 109
7 more
  1. 2024FlightLLM: Efficient Large Language Model Inference with a Complete Mapping Flow on FPGAscited by 64
  2. 2023Understanding the Potential of FPGA-Based Spatial Acceleration for Large Language Model Inferencecited by 33
  3. 2018BISMO: A Scalable Bit-Serial Matrix Multiplication Overlay for Reconfigurable Computingcited by 22
  4. 2021HiKonv: High Throughput Quantized Convolution With Novel Bit-wise Management and Computationcited by 6
  5. 2023M4BRAM: Mixed-Precision Matrix-Matrix Multiplication in FPGA Block RAMscited by 5
  6. 2024FlexiBit: Fully Flexible Precision Bit-parallel Accelerator Architecture for Arbitrary Mixed Precision AIcited by 5
  7. 2024TATAA: Programmable Mixed-Precision Transformer Acceleration with a Transformable Arithmetic Architecturecited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026TileFuse: A Fused Mixed-Precision Kernel Library for Efficient Quantized LLM Inference on AMD NPUs
  2. 2023A Dataflow Compiler for Efficient LLM Inference using Custom Microscaling Formats
  3. 2025FPGA Co-Design for Efficient N:M Sparse and Quantized Model Inference
  4. 2024MixPE: Quantization and Hardware Co-design for Efficient LLM Inference
  5. 2024T-MAC: CPU Renaissance via Table Lookup for Low-Bit LLM Deployment on Edge
  6. 2025LUT-LLM: Efficient Large Language Model Inference with Memory-based Computations on FPGAs
  7. 2026Hardware Generation and Exploration of Lookup Table-Based Accelerators for 1.58-bit LLM Inference
  8. 2024LlamaF: An Efficient Llama2 Architecture Accelerator on Embedded FPGAs