mlsanityData to 7 Oct 2026

HeatKV: Head-tuned KV-cache Compression for Visual Autoregressive Modeling

arXiv 2605.14877 ↗First posted 14 May 2026cs.CVcited by 0

On the map: Efficient LLM inference and serving → Fast autoregressive visual generation

Abstract (arXiv)

Visual Autoregressive (VAR) models have recently demonstrated impressive image generation quality while maintaining low latency. However, they suffer from severe KV-cache memory constraints, often requiring gigabytes of memory per generated image. We introduce HeatKV, a novel compression method that adapts cache allocation in each head based on its attention to previously generated scales. Using a small offline calibration set, the attention heads are ranked according to their attention scores over prior scales. Based on this ranking, we construct a static pruning schedule tailored to a given memory budget. Applied to the Infinity-2B model, HeatKV achieves 2 × higher compression ratio in memory allocation for KV cache compared to existing methods, while maintaining similar or better image fidelity, prompt alignment and human perception score. Our method achieves a new state-of-the-art (SOTA) for VAR model KV-cache compression, showcasing the effectiveness of fine-grained, head-specific cache allocation. Code and calibration script available at https://github.com/arm-research/heatkv.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Fast autoregressive visual generation, one of 12 directions in Efficient LLM inference and serving. Builds on 37 of its 40 references that are in the map. First posted on arXiv 14 May 2026.

Builds on

37 of 40 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025AMS-KV: Adaptive KV Caching in Multi-Scale Visual Autoregressive Transformers20 Nov
  2. 2025Memory-Efficient Visual Autoregressive Modeling with Scale-Aware KV Cache Compression26 May
  3. 2025Head-Aware KV Cache Compression for Efficient Visual Autoregressive Modeling12 Apr
  4. 2025FastVAR: Linear Visual Autoregressive Modeling via Cached Token Pruning30 Mar

Other references · 33

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2025Qwen3 Technical Reportcited by 8,195
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  6. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  7. 2022Scalable Diffusion Models with Transformerscited by 5,931
  8. 2017Neural Discrete Representation Learningcited by 4,650
  9. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  10. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  11. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  12. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
21 more
  1. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
  2. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
  3. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 848
  4. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 847
  5. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 805
  6. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 760
  7. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 633
  8. 2022StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasetscited by 489
  9. 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 297
  10. 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inferencecited by 230
  11. 2024HART: Efficient Visual Generation with Hybrid Autoregressive Transformercited by 128
  12. 2024Image and Video Tokenization with Binary Spherical Quantizationcited by 110
  13. 2024FlowAR: Scale-wise Autoregressive Image Generation Meets Flow Matchingcited by 77
  14. 2024Parallelized Autoregressive Visual Generationcited by 58
  15. 2025InfinityStar: Unified Spacetime AutoRegressive Modeling for Visual Generationcited by 43
  16. 2025Visual Autoregressive Modeling for Image Super-Resolutioncited by 36
  17. 2025FlexVAR: Flexible Visual Autoregressive Modeling without Residual Predictioncited by 22
  18. 2024LiteVAR: Compressing Visual Autoregressive Modelling with Efficient Attention and Quantizationcited by 16
  19. 2026SparVAR: Exploring Sparsity in Visual AutoRegressive Modeling for Training-Free Accelerationcited by 4
  20. 2026PTQ4ARVG: Post-Training Quantization for AutoRegressive Visual Generation Modelscited by 3
  21. 2025Visual Autoregressive Modelling for Monocular Depth Estimationcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Head-Aware Key-Value Compression for Efficient Autoregressive Image Generation
  2. 2025Visual Autoregressive Transformers Must Use Omega(n2 d) Memory
  3. 2026HACK++: Towards More Effective Head-Aware Key-Value Compression for Efficient Visual Autoregressive Modeling
  4. 2025Autoregressive Image Generation Needs Only a Few Lines of Cached Tokens
  5. 2025SSD: Spatial-Semantic Head Decoupling for Efficient Autoregressive Image Generation
  6. 2026Forcing-KV: Hybrid KV Cache Compression for Efficient Autoregressive Video Diffusion Models
  7. 2026PackCache: A Training-Free Acceleration Method for Unified Autoregressive Video Generation via Compact KV-Cache
  8. 2026Future Forcing: Future-aware Training-free KV Cache Policy for Autoregressive Video Generation