mlsanityData to 14 Oct 2026

LOOKAT: Lookup-Optimized Key-Attention for Memory-Efficient Transformers

arXiv 2601.10155 ↗First posted 15 Jan 2026cs.LG cs.AIcited by 4

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

Compressing the KV cache is a required step to deploy large language models on edge devices. Current quantization methods compress storage but fail to reduce bandwidth as attention calculation requires dequantizing keys from INT4/INT8 to FP16 before use. We observe that attention scoring is mathematically equivalent to the inner product similarity search and we can apply some compression techniques from vector databases to compress KV-cache better. We propose LOOKAT, which applies product quantization and asymmetric distance computation, to transformer architecture by decomposing key vectors into subspaces, learning codebooks and computing attention tables via lookup tables. This transforms attention from memory-bound to compute-bound. LOOKAT achieves 64 × compression at 95.7% output fidelity and 32 × compression at 95.0% fidelity when tested on GPT-2. LOOKAT requires no architecture changes or training while maintaining rank correlation ρ > 0.95. Theoretical analysis confirms that rank correlation degrades as O(dk/mK), with guarantees validated across sequence lengths up to 1024 tokens.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 16 of its 20 references that are in the map. First posted on arXiv 15 Jan 2026.

Builds on

16 of 20 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models24 Jun

Other references · 15

  1. 2017Attention Is All You Needcited by 60,361
  2. 2017Billion-scale similarity search with GPUscited by 2,497
  3. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  4. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  5. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  6. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  7. 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 861
  8. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 717
  9. 2020Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuningcited by 671
  10. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 637
  11. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 575
  12. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 367
3 more
  1. 2023Atom: Low-bit Quantization for Efficient and Accurate LLM Servingcited by 272
  2. 2023OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Modelscited by 114
  3. 2023ZeroQuant(4+2): Redefining LLMs Quantization with a New FP6-Centric Strategy for Diverse Generative Taskscited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026AXELRAM: Quantize Once, Never Dequantizecited by 1
  2. 2026HeadQ: Model-Visible Distortion and Score-Space Correction for KV-Cache Quantizationcited by 0
  3. 2026SCOPE: Subspace Clustering with Online Per-Head Top-K Estimation for Sparse Video Attentioncited by 0
  4. 2026E8 Lattice Key Quantization for Low-Bit KV-Cache Compressioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Bridging KV-Cache Quantization and Linear Attention: From Theory to Pretrained Weight Migration
  2. 2026Keyless Attention: Value-Space Routing and Value-Only Caching for Efficient Transformers
  3. 2026KV Cache Compression Through the Lens of Transform Coding
  4. 2025End-to-End Transformer Acceleration Through Processing-in-Memory Architectures
  5. 2025HCAttention: Extreme KV Cache Compression via Heterogeneous Attention Computing for LLMs
  6. 2026Tensor Cache: Eviction-conditioned Associative Memory for Transformers
  7. 2025EdgeInfinite: A Memory-Efficient Infinite-Context Transformer for Edge Devices
  8. 2026Low-Rank Key Value Attention