mlsanityData to 7 Oct 2026

HACK: Homomorphic Acceleration via Compression of the Key-Value Cache for Disaggregated LLM Inference

arXiv 2502.03589 ↗First posted 5 Feb 2025cs.DC cs.LGcited by 10

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

Disaggregated Large Language Model (LLM) inference has gained popularity as it separates the computation-intensive prefill stage from the memory-intensive decode stage, avoiding the prefill-decode interference and improving resource utilization. However, transmitting Key-Value (KV) data between the two stages can be a bottleneck, especially for long prompts. Additionally, the computation time overhead for prefill and decode is key for optimizing Job Completion Time (JCT), and KV data size can become prohibitive for long prompts and sequences. Existing KV quantization methods can alleviate the transmission bottleneck and reduce memory requirements, but they introduce significant dequantization overhead, exacerbating the computation time. We propose Homomorphic Acceleration via Compression of the KV cache (HACK) for disaggregated LLM inference. HACK eliminates the heavy KV dequantization step, and directly performs computations on quantized KV data to approximate and reduce the cost of the expensive matrix-multiplication step. Extensive trace-driven experiments show that HACK reduces JCT by up to 70.9% compared to disaggregated LLM inference baseline and by up to 52.3% compared to state-of-the-art KV quantization methods.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 10 of its 17 references that are in the map. First posted on arXiv 5 Feb 2025.

Builds on

10 of 17 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024D'ej`aVu: KV-cache Streaming for Fast, Fault-tolerant Generative LLM Serving4 Mar
  2. 2023CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving11 Oct

Other references · 8

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  2. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 595
  3. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 571
  4. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 536
  5. 2018A Discourse-Aware Attention Model for Abstractive Summarization of Long Documentscited by 477
  6. 2024Mooncake: A KVCache-centric Disaggregated Architecture for LLM Servingcited by 193
  7. 2024Inference without Interference: Disaggregate LLM Inference for Mixed Downstream Workloadscited by 120
  8. 2024Cocktail: A Comprehensive Information Retrieval Benchmark with LLM-Generated Documents Integrationcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 9.

10 papers.

  1. 2026KVServe: Service-Aware KV Cache Compression for Communication-Efficient Disaggregated LLM Servingcited by 7
  2. 2026DynamiQ: Accelerating Gradient Synchronization using Compressed Multi-hop All-reducecited by 4
  3. 2026Quantizing With Randomized Hadamard Transforms: Efficient Heuristic Now Provencited by 4
  4. 2026Runtime-Certified Bounded-Error Quantized Attentioncited by 4
  5. 2026SmartGen: Seamless Disaggregated LLM Inference with Selective KV Cache Transfercited by 2
  6. 2026Lynx: Progressive Speculative Quantization for accelerating KV Transfer in Long-Context Inferencecited by 1
  7. 2026Entropy-Constrained Adaptive Stochastic Quantizationcited by 1
  8. 2025UltraSketchLLM: Sub-1-Bit LLM Compression via Sketch and Hardware-Friendly Operatorscited by 0
  9. 2026Unified AI Gateway: A Framework for Joint Model Routing and KV Cache Managementcited by 0
  10. 2026TORQUE: Optimizing What (not) to Quantize Before and After Rotationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Fast KVzip: Efficient and Accurate LLM Inference with Gated KV Eviction
  2. 2025VecInfer: Efficient LLM Inference with Low-Bit KV Cache via Outlier-Suppressed Vector Quantization
  3. 2024Get More with LESS: Synthesizing Recurrence with KV Cache Compression for Efficient LLM Inference
  4. 2024GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLM
  5. 2024KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantization
  6. 2026KV Cache Optimization Strategies for Scalable and Efficient LLM Inference
  7. 2025HAMburger: Accelerating LLM Inference via Token Smashing
  8. 2024KVPR: Efficient LLM Inference with I/O-Aware KV Cache Partial Recomputation