mlsanityData to 14 Oct 2026

Last But Not Least: Boundary Attention CalibratiON for Multimodal KV Cache Compression

arXiv 2606.14782 ↗First posted 10 Jun 2026cs.CV cs.CLcited by 1

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 2 more
Abstract (arXiv)

Multimodal Large Language Models (MLLMs) achieve strong vision-language reasoning but incur large KV caches and high decoding latency with long visual contexts. Existing compression methods rely on observation window attention for stable token importance estimation, yet this aggregation can dilute sparse critical evidence and discard answer-relevant tokens under aggressive compression. We identify last query attention as a complementary signal for recovering such evidence, though its irrelevant signals may introduce additional noise. We propose BACON, a plug-and-play method that calibrates observation window attention with last query evidence while suppressing noise through intra-layer coherence and inter-layer persistence. Across diverse benchmarks, models, budgets, and compression methods, BACON improves multimodal KV-cache compression by 7.5% on average under the most aggressive budget, with gains up to 30.9%.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 23 of its 27 references that are in the map. First posted on arXiv 10 Jun 2026.

Builds on

23 of 27 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Not All Heads Matter: A Head-Level KV Cache Compression Method with Integrated Retrieval and Reasoning25 Oct

Other references · 22

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  2. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  3. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  4. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  5. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  6. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  7. 2019Towards VQA Models That Can Readcited by 1,718
  8. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  9. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  10. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  11. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  12. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 880
10 more
  1. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 744
  2. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 432
  3. 2019VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Researchcited by 414
  4. 2020TextCaps: a Dataset for Image Captioning with Reading Comprehensioncited by 400
  5. 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Usecited by 254
  6. 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inferencecited by 235
  7. 2024LOOK-M: Look-Once Optimization in KV Cache for Efficient Multimodal Long-Context Inferencecited by 95
  8. 2025KVzip: Query-Agnostic KV Cache Compression with Context Reconstructioncited by 95
  9. 2025InfiniPot-V: Memory-Constrained KV Cache Compression for Streaming Video Understandingcited by 58
  10. 2025SparseMM: Head Sparsity Emerges from Visual Concept Responses in MLLMscited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Seen, Said, or Forgotten? A Causal Audit of Visual KV Memory Across Dialog Turnscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026HybridKV: Hybrid KV Cache Compression for Efficient Multimodal Large Language Model Inference
  2. 2026PuzzleKV: Page-Wise Low-Rank Decomposition for KV Cache Compression
  3. 2025Expected Attention: KV Cache Compression by Estimating Attention from Future Queries Distribution
  4. 2026SGD-KV: Summarization Guided KV Cache Compression
  5. 2026Moment-KV: Momentum-Based Decode-Time KV Cache Compression for Long Generation
  6. 2025SALS: Sparse Attention in Latent Space for KV cache Compression
  7. 2025MEDA: Dynamic KV Cache Allocation for Efficient Multimodal Long-Context Inference
  8. 2025SmallKV: Small Model Assisted Compensation of KV Cache Compression for Efficient LLM Inference