mlsanityData to 7 Oct 2026

LogQuant: Log-Distributed 2-Bit Quantization of KV Cache with Superior Accuracy Preservation

arXiv 2503.19950 ↗First posted 25 Mar 2025cs.LG cs.AI cs.CLcited by 0

On the map: Efficient LLM inference and serving → KV cache quantization

Abstract (arXiv)

We introduce LogQuant, a groundbreaking 2-bit quantization technique for KV Cache in large language model (LLM) inference, delivering substantial memory savings while preserving superior performance. Previous methods either assume that later tokens are more important or attempt to predict important tokens based on earlier attention patterns. Both approaches, however, can result in performance bottlenecks or frequent mispredictions. LogQuant takes a different approach. By applying a log-based filtering mechanism, it selectively compresses the KV Cache across the entire context, achieving better performance with the same or even reduced memory footprint compared to existing methods. In benchmark tests, it enhances throughput by 25% and boosts batch size by 60% without increasing memory consumption. For challenging tasks such as Math and Code Completion, LogQuant improves accuracy by 40% to 200% at the same compression ratio, outperforming comparable techniques.LogQuant integrates effortlessly with popular inference frameworks like Python's transformers library. Implementation can be available in https://github.com/Concyclics/LogQuantKV.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in KV cache quantization, one of 12 directions in Efficient LLM inference and serving. Builds on 22 of its 29 references that are in the map. First posted on arXiv 25 Mar 2025.

Builds on

22 of 29 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving7 May
  2. 2024QAQ: Quality Adaptive Quantization for LLM KV Cache7 Mar
  3. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache5 Feb

Other references · 19

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2023Qwen Technical Reportcited by 3,273
  5. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  6. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  7. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  8. 2024Qwen2 Technical Reportcited by 2,076
  9. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  10. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  11. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  12. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
7 more
  1. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 848
  2. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 805
  3. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 700
  4. 2024MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attentioncited by 375
  5. 2024GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLMcited by 155
  6. 2024LLM Inference Unveiled: Survey and Roofline Model Insightscited by 147
  7. 2024Keyformer: KV Cache Reduction through Key Tokens Selection for Efficient Generative Inferencecited by 124

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization
  2. 2025XQuant: Achieving Ultra-Low Bit KV Cache Quantization with Cross-Layer Compression
  3. 2025NSNQuant: A Double Normalization Approach for Calibration-Free Low-Bit Vector Quantization of KV Cache
  4. 2025XQuant: Breaking the Memory Wall for LLM Inference with KV Cache Rematerialization
  5. 2025Accurate KV Cache Quantization with Outlier Tokens Tracing
  6. 2025CalibQuant: 1-Bit KV Cache Quantization for Multimodal LLMs
  7. 2025Kitty: Accurate and Efficient 2-bit KV Cache Quantization with Dynamic Channel-wise Precision Boost
  8. 2025NQKV: A KV Cache Quantization Scheme Based on Normal Distribution Characteristics