mlsanityData to 14 Oct 2026

Lexico: Extreme KV Cache Compression via Sparse Coding over Universal Dictionaries

arXiv 2412.08890 ↗First posted 12 Dec 2024cs.LGcited by 17

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

We introduce Lexico, a novel KV cache compression method that leverages sparse coding with a universal dictionary. Our key finding is that key-value cache in modern LLMs can be accurately approximated using sparse linear combination from a small, input-agnostic dictionary of ~4k atoms, enabling efficient compression across different input prompts, tasks and models. Using orthogonal matching pursuit for sparse approximation, Lexico achieves flexible compression ratios through direct sparsity control. On GSM8K, across multiple model families (Mistral, Llama 3, Qwen2.5), Lexico maintains 90-95% of the original performance while using only 15-25% of the full KV-cache memory, outperforming both quantization and token eviction methods. Notably, Lexico remains effective in low memory regimes where 2-bit quantization fails, achieving up to 1.7x better compression on LongBench and GSM8K while maintaining high accuracy.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 32 of its 40 references that are in the map. First posted on arXiv 12 Dec 2024.

Builds on

32 of 40 references.

Closest ancestors · 12

Same direction, newest first

  1. 2024Model Tells You Where to Merge: Adaptive KV Cache Merging for LLMs on Long-Context Tasks11 Jul
  2. 2024A Simple and Effective L2 Norm-Based Strategy for KV Cache Compression17 Jun
  3. 2024Effectively Compress KV Heads for LLM11 Jun
  4. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling4 Jun
  5. 2024SnapKV: LLM Knows What You are Looking for Before Generation22 Apr
  6. 2024GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLM8 Mar
  7. 2024ChunkAttention: Efficient Self-Attention with Prefix-Aware KV Cache and Two-Phase Partition23 Feb
  8. 2024Get More with LESS: Synthesizing Recurrence with KV Cache Compression for Efficient LLM Inference14 Feb
  9. 2023Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs3 Oct
  10. 2023Efficient Memory Management for Large Language Model Serving with PagedAttention12 Sep
  11. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models24 Jun
  12. 2023Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time26 May

Other references · 20

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2020Longformer: The Long-Document Transformercited by 3,413
  4. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  5. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  6. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  7. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  8. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 867
  9. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 717
  10. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 637
  11. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 591
  12. 2013k-Sparse Autoencoderscited by 215
8 more
  1. 2024Mooncake: A KVCache-centric Disaggregated Architecture for LLM Servingcited by 196
  2. 2024You Only Cache Once: Decoder-Decoder Architectures for Language Modelscited by 144
  3. 2024ZipCache: Accurate and Efficient KV Cache Quantization with Salient Token Identificationcited by 93
  4. 2024Loki: Low-rank Keys for Efficient Sparse Attentioncited by 89
  5. 2024Infinite-LLM: Efficient LLM Service for Long Context with DistAttention and Distributed KVCachecited by 73
  6. 2024WKVQuant: Quantizing Weight and Key/Value Cache for Large Language Models Gains Morecited by 69
  7. 2014From Denoising to Compressed Sensingcited by 68
  8. 2024A Practical GPU-Accelerated Implementation of Orthogonal Matching Pursuitcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 10; 2026: 7.

17 papers.

Sort:

By area:

  1. 2025TurboQuant: Online Vector Quantization with Near-optimal Distortion Ratecited by 125
  2. 2025Cartridges: Lightweight and general-purpose long context representations via self-studycited by 86
  3. 2025Is Random Attention Sufficient for Sequence Modeling? Disentangling Trainable Components in the Transformercited by 12
  4. 2025KV Cache Compression for Inference Efficiency in LLMs: A Reviewcited by 12
  5. 2025Streaming Attention Approximation via Discrepancy Theorycited by 11
  6. 2026ResidualKV: Residual-Based KV Cache Compression for Efficient Long-Context Inferencecited by 9
  7. 2025SQuat: Subspace-orthogonal KV Cache Quantizationcited by 7
  8. 2026Training Transformers for KV Cache Compressibilitycited by 7
  9. 2025CoSpaDi: Compressing LLMs via Calibration-Guided Sparse Dictionary Learningcited by 6
  10. 2025Compressed Convolutional Attention: Efficient Attention in a Compressed Latent Spacecited by 5
  11. 2026Towards Fast and Effective Long Video Understanding of Multimodal Large Language Models via Adaptive Quasi-Gaussian Samplingcited by 5
  12. 2026OCTOPUS: Optimized KV Cache for Transformers via Octahedral Parametrization Under optimal Squared error quantizationcited by 2
5 more
  1. 2025SWAN: Sparse Winnowed Attention for Reduced Inference Memory via Decompression-Free KV-Cache Compressioncited by 1
  2. 2025Unlocking the Address Book: Dissecting the Sparse Semantic Structure of LLM Key-Value Caches via Sparse Autoencoderscited by 1
  3. 2026Linear Predictability of Attention Heads in Large Language Modelscited by 1
  4. 2026LLM Inference in a Flash!cited by 1
  5. 2026OMP-MoE: Efficient Expert Pruning for Mixture-of-Experts LLMs via Orthogonal Matching Pursuitcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Compactor: Calibrated Query-Agnostic KV Cache Compression with Approximate Leverage Scores
  2. 2025Ecco: Improving Memory Bandwidth and Capacity for LLMs via Entropy-aware Cache Compression
  3. 2024CSR:Achieving 1 Bit Key-Value Cache via Sparse Representation
  4. 2026Sequential KV Cache Compression via Probabilistic Language Tries: Beyond the Per-Vector Shannon Limit
  5. 2026EchoKV: Efficient KV Cache Compression via Similarity-Based Reconstruction
  6. 2026The risk of KV cache compression
  7. 2026GRKV: Global Regression for Training-Free KV Cache Compression in Long-Context LLMs
  8. 2026KV-CoRE: Benchmarking Data-Dependent Low-Rank Compressibility of KV-Caches in LLMs