mlsanityData to 7 Oct 2026

XQuant: Achieving Ultra-Low Bit KV Cache Quantization with Cross-Layer Compression

arXiv 2510.11236 ↗First posted 13 Oct 2025cs.CLcited by 4

On the map: Efficient LLM inference and serving → KV cache quantization

Abstract (arXiv)

Large Language Models (LLMs) have demonstrated remarkable capabilities across diverse natural language processing tasks. However, their extensive memory requirements, particularly due to KV cache growth during long-text understanding and generation, present significant challenges for deployment in resource-constrained environments. Quantization has emerged as a promising solution to reduce memory consumption while preserving historical information. We propose XQuant, a training-free and plug-and-play framework that achieves ultra-low equivalent bit-width KV cache quantization. XQuant introduces two key innovations: a computationally negligible data-free calibration method and cross-layer KV cache compression, enabling quantization to sub-1.4 bits. Extensive experiments on TruthfulQA and LongBench demonstrate that XQuant outperforms state-of-the-art methods (e.g., KIVI-2bit and AsymKV-1.5bit) by achieving lower bit-width while maintaining superior performance, establishing a better trade-off between memory efficiency and model accuracy.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in KV cache quantization, one of 12 directions in Efficient LLM inference and serving. Builds on 22 of its 35 references that are in the map. First posted on arXiv 13 Oct 2025.

Builds on

22 of 35 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024AsymKV: Enabling 1-Bit Quantization of KV Cache with Layer-Wise Asymmetric Quantization Configurations17 Oct
  2. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache5 Feb
  3. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization31 Jan

Other references · 19

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  3. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  4. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  5. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  6. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  7. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  8. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  9. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 848
  10. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 567
  11. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 420
  12. 2024PyramidInfer: Pyramid KV Cache Compression for High-throughput LLM Inferencecited by 148
7 more
  1. 2024You Only Cache Once: Decoder-Decoder Architectures for Language Modelscited by 135
  2. 2024Reducing Transformer Key-Value Cache Size with Cross-Layer Attentioncited by 122
  3. 2024No Token Left Behind: Reliable KV Cache Compression via Importance-Aware Mixed Precision Quantizationcited by 99
  4. 2024Keep the Cost Down: A Review on Methods to Optimize LLM' s KV-Cache Consumptioncited by 96
  5. 2024KVSharer: Efficient Inference via Layer-Wise Dissimilar KV Cache Sharingcited by 38
  6. 2025SpindleKV: A Novel KV Cache Reduction Method Balancing Both Shallow and Deep Layerscited by 9
  7. 2025KV-Latent: Dimensional-level KV Cache Reduction with Frequency-aware Rotary Positional Embeddingcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2025Ghost in the Transformer: Detecting Model Reuse with Invariant Spectral Signaturescited by 2
  2. 2026From AR to Diffusion: Efficiently Adapting Large Language Models with Strictly Causal and Elastic Horizonscited by 1
  3. 2026CoQuant: Joint Weight-Activation Subspace Projection for Mixed-Precision LLMscited by 0
  4. 2026OffQ: Taming Structured Outliers in LLM Quantization by Offsettingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024WKVQuant: Quantizing Weight and Key/Value Cache for Large Language Models Gains More
  2. 2026InnerQ: Hardware-Aware Tuning-Free Quantization of KV Cache for Large Language Models
  3. 2025CalibQuant: 1-Bit KV Cache Quantization for Multimodal LLMs
  4. 2025NQKV: A KV Cache Quantization Scheme Based on Normal Distribution Characteristics
  5. 2024KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantization
  6. 2026OScaR: The Occam's Razor for Extreme KV Cache Quantization in LLMs and Beyond
  7. 2024Unlocking Data-free Low-bit Quantization with Matrix Decomposition for KV Cache Compression
  8. 2025PM-KVQ: Progressive Mixed-precision KV Cache Quantization for Long-CoT LLMs