AXELRAM: Quantize Once, Never Dequantize
On the map: Efficient LLM inference and serving → KV cache quantization
Abstract (arXiv)
We propose AXELRAM, a smart SRAM macro architecture that computes attention scores directly from quantized KV cache indices without dequantization. The key enabler is a design-time fixed codebook: orthogonal-transform-based quantization concentrates each coordinate's distribution to N(0,1/d), so the optimal quantizer depends only on dimension d and bit-width b, not on input data. The asymmetric path design -- transform on write, table-lookup on read with no inverse transform -- reduces per-query multiplications by 102.4x (a mathematical identity). Through multi-seed evaluation (10 seeds x 3 models), we discover that sign pattern sensitivity causes catastrophic PPL spikes (Delta > 50) on certain models (Qwen2.5-3B), while others (LLaMA-3.1-8B) are fully stable. This phenomenon extends SpinQuant's observation of rotation variance in weight quantization to the KV cache domain, where the effect is qualitatively more severe. We trace the root cause to layer-wise norm heterogeneity and propose a gradient-free sign pattern selection (200 candidates, 8 calibration samples, one-time) that eliminates catastrophic spikes with zero additional hardware cost. All source code is available at https://github.com/Axelidea/AXELRAM.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in KV cache quantization, one of 12 directions in Efficient LLM inference and serving. Builds on 8 of its 19 references that are in the map. First posted on arXiv 3 Apr 2026.
Builds on
8 of 19 references.
Closest ancestors · 4
Same direction, newest first
- 2024QJL: 1-Bit Quantized JL Transform for KV Cache Quantization with Zero Overhead5 Jun
- 2024SpinQuant: LLM quantization with learned rotations26 May
- 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs30 Mar
- 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache5 Feb
Other references · 4
- 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 354
- 2025TurboQuant: Online Vector Quantization with Near-optimal Distortion Ratecited by 116
- 2021Multiplying Matrices Without Multiplyingcited by 30
- 2026LOOKAT: Lookup-Optimized Key-Attention for Memory-Efficient Transformerscited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026RoPE-Aware Bit Allocation for KV-Cache Quantization
- 2026SAW-INT4: System-Aware 4-Bit KV-Cache Quantization for Real-World LLM Serving
- 2026OScaR: The Occam's Razor for Extreme KV Cache Quantization in LLMs and Beyond
- 2026TurboAngle: Near-Lossless KV Cache Compression via Uniform Angle Quantization
- 2026RateQuant: Optimal Mixed-Precision KV Cache Quantization via Rate-Distortion Theory
- 2024AsymKV: Enabling 1-Bit Quantization of KV Cache with Layer-Wise Asymmetric Quantization Configurations
- 2026Bridging KV-Cache Quantization and Linear Attention: From Theory to Pretrained Weight Migration
- 2026QMC: Efficient SLM Edge Inference via Outlier-Aware Quantization and Emergent Memories Co-Design