mlsanityData to 14 Oct 2026

LiquidGEMM: Hardware-Efficient W4A8 GEMM Kernel for High-Performance LLM Serving

arXiv 2509.01229 ↗First posted 1 Sep 2025cs.DC cs.AI cs.LGcited by 8

On the map: Efficient LLM inference and serving → KV cache quantization

Abstract (arXiv)

Quantization is a critical technique for accelerating LLM inference by reducing memory footprint and improving computational efficiency. Among various schemes, 4-bit weight and 8-bit activation quantization (W4A8) offers a strong balance between accuracy and performance. However, existing W4A8 GEMM kernels fall short in practice due to inefficient dequantization on CUDA Cores, which cannot keep pace with the high throughput of Tensor Cores. In this paper, we present LiquidGEMM, a hardware-efficient W4A8 GEMM kernel for efficient LLM serving. LiquidGEMM designs two key techniques: LiquidQuant, a hardware-efficient quantization method that enables fast, overflow-safe dequantization using just two arithmetic instructions per four elements; and an implicit fine-grained pipeline that fully overlaps weight loading, dequantization, and MMA across warp groups without software synchronization or redundant memory traffic. Experimental results show that LiquidGEMM achieves up to 2.90x speedup over state-of-the-art W4A8 kernels and up to 4.94x end-to-end system-level speedup. Compared to various quantized GEMM kernels in NVIDIA TensorRT-LLM, LiquidGEMM delivers 1.12-1.63x performance gains, and achieves up to 1.63x system-level speedup.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in KV cache quantization, one of 12 directions in Efficient LLM inference and serving. Builds on 20 of its 30 references that are in the map. First posted on arXiv 1 Sep 2025.

Builds on

20 of 30 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024SpinQuant: LLM quantization with learned rotations26 May
  2. 2024QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving7 May
  3. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs30 Mar
  4. 2023Atom: Low-bit Quantization for Efficient and Accurate LLM Serving29 Oct

Other references · 16

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  5. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  6. 2016Pointer Sentinel Mixture Modelscited by 3,096
  7. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  8. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  9. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  10. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  11. 2024Yi: Open Foundation Models by 01.AIcited by 719
  12. 2024FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisioncited by 519
4 more
  1. 2019WinoGrande: An Adversarial Winograd Schema Challenge at Scalecited by 204
  2. 2024DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMscited by 167
  3. 2024Low-Rank Quantization-Aware Training for LLMscited by 43
  4. 2024COMET: Towards Partical W4A4KV4 LLMs Servingcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 5.

8 papers.

  1. 2025CudaForge: An Agent Framework with Hardware Feedback for CUDA Kernel Optimizationcited by 45
  2. 2025Parallel Loop Transformer for Efficient Test-Time Computation Scalingcited by 24
  3. 2025LayerScope: Predictive Cross-Layer Scheduling for Efficient Multi-Batch MoE Inference on Legacy Serverscited by 8
  4. 2026The Integer Alibi: Localizing Cross-Kernel Divergence in INT8-Quantized LLM Inferencecited by 2
  5. 2026Multi-Scale Dequant: Eliminating Dequantization Bottleneck via Activation Decomposition for Efficient LLM Inferencecited by 1
  6. 2026APEX4: Efficient Pure W4A4 LLM Inference via Intra-SM Compute Rebalancingcited by 1
  7. 2026Realizing Native INT8 Compute for Diffusion Transformers on Consumer GPUs: A Fused INT8 GEMM Kernel for Ideogram 4.0cited by 0
  8. 2026ExaGEMM: Exploration Framework for CPU-Driven ML Inference via Associative In-Register Computing for Low-Bit GEMMcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025CodeGEMM: A Codebook-Centric Approach to Efficient GEMM in Quantized LLMs
  2. 2024LLMEasyQuant: Scalable Quantization for Parallel and Distributed LLM Inference
  3. 2026TileFuse: A Fused Mixed-Precision Kernel Library for Efficient Quantized LLM Inference on AMD NPUs
  4. 2024LUT Tensor Core: A Software-Hardware Co-Design for LUT-Based Low-Bit LLM Inference
  5. 2025APT-LLM: Exploiting Arbitrary-Precision Tensor Core Computing for LLM Acceleration
  6. 2024Accurate Block Quantization in LLMs with Outliers
  7. 2024QUICK: Quantization-aware Interleaving and Conflict-free Kernel for efficient LLM inference
  8. 2025Pushing the Envelope of LLM Inference with Ultra-Low-Bit Quantized Models