mlsanityData to 14 Oct 2026

PacQ: A SIMT Microarchitecture for Efficient Dataflow in Hyper-asymmetric GEMMs

arXiv 2502.18627 ↗First posted 25 Feb 2025cs.ARcited by 0

Not on the map. Why?

Abstract (arXiv)

Weight-only quantization has been widely explored in large language models (LLMs) to reduce memory storage and data loading overhead. During deployment on single-instruction-multiple-threads (SIMT) architectures, weights are stored in low-precision integer (INT) format, while activations remain in full-precision floating-point (FP) format to preserve inference accuracy. Although memory footprint and data loading requirements for weight matrices are reduced, computation performance gains remain limited due to the need to convert weights back to FP format through unpacking and dequantization before GEMM operations. In this work, we investigate methods to accelerate GEMM operations involving packed low-precision INT weights and high-precision FP activations, defining this as the hyper-asymmetric GEMM problem. Our approach co-optimizes tile-level packing and dataflow strategies for INT weight matrices. We further design a specialized FP-INT multiplier unit tailored to our packing and dataflow strategies, enabling parallel processing of multiple INT weights. Finally, we integrate the packing, dataflow, and multiplier unit into PacQ, a SIMT microarchitecture designed to efficiently accelerate hyper-asymmetric GEMMs. We show that PacQ can achieve up to 1.99x speedup and 81.4% reduction in EDP compared to weight-only quantized LLM workloads running on conventional SIMT baselines.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 14 of its 24 references that are in the map. First posted on arXiv 25 Feb 2025.

Builds on

14 of 24 references.

References · 14

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  2. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  3. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  4. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  5. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  6. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  7. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 479
  8. 2023Atom: Low-bit Quantization for Efficient and Accurate LLM Servingcited by 272
  9. 2023BitNet: Scaling 1-bit Transformers for Large Language Modelscited by 189
  10. 2022LUT-GEMM: Quantized Matrix Multiplication based on LUTs for Efficient Inference in Large-Scale Generative Language Modelscited by 105
  11. 2022Who Says Elephants Can't Run: Bringing Large Scale MoE Models into Cloud Scale Productioncited by 27
  12. 2018Modeling Deep Learning Accelerator Enabled GPUscited by 10
2 more
  1. 2024TesseraQ: Ultra Low-Bit LLM Post-Training Quantization with Block Reconstructioncited by 4
  2. 2020High Throughput Matrix-Matrix Multiplication between Asymmetric Bit-Width Operandscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025FIGLUT: An Energy-Efficient Accelerator Design for FP-INT GEMM Using Look-Up Tables
  2. 2025CodeGEMM: A Codebook-Centric Approach to Efficient GEMM in Quantized LLMs
  3. 2025The Cambrian Explosion of Mixed-Precision Matrix Multiplication for Quantized Deep Learning Inference
  4. 2026MxGLUT: A Reconfigurable LUT-Centric Broadcast Dataflow Accelerator for Mixed-Precision GEMM ↗
  5. 2025Platinum: Path-Adaptable LUT-Based Accelerator Tailored for Low-Bit Weight Matrix Multiplication
  6. 2025MEADOW: Memory-efficient Dataflow and Data Packing for Low Power Edge LLMs
  7. 2026ExaGEMM: Exploration Framework for CPU-Driven ML Inference via Associative In-Register Computing for Low-Bit GEMM
  8. 2025Accelerating Sparse Ternary GEMM for Quantized ML on Apple Silicon