Pixel Decodability Is Not a Compression Signal: Causally Evaluating Importance Proxies for Visual KV-Cache Eviction
On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency
Abstract (arXiv)
Vision-language models retain a substantial amount of pixel-decodable visual content in their visual key-value cache. We show, in our setting, that this retention is task-inert: across our preregistered tests, how much a unit retains never positively tracks whether the computation that answers the question causally relies on it. We measure retention with a learned pixel-inversion decoder and causal use with single-super-patch KV ablation, the teacher-forced drop in gold-answer log-probability, and relate the two within images under a preregistered, sign-calibrated, held-out design. Retention is decoupled from attention and, in a well-powered null, from causal utilization. Utilization is not inert to every proxy: attention weakly but significantly tracks it, the only signal we find that does and the design's positive control. We characterize pixel-decodable retention as an informational axis of the visual KV cache, orthogonal to the functional one. How much task-inert content a cache holds differs by architecture in our model pair: the encoder-free model retains 2.7 times more than the encoder-based one. The engineering consequence is a controlled negative result. At super-patch granularity, deconfounded pixel-decodable retention ranks KV eviction no better than random; at token granularity it acquires only a weak inverse-importance signal at larger budgets, dominated at every budget by attention magnitude. In our setting, pixel-decodable reconstructability is not a competitive KV-compression signal at any granularity we test.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 12 of its 13 references that are in the map. First posted on arXiv 27 Jul 2026.
Builds on
12 of 13 references.
Closest ancestors · 6
Same direction, newest first
- 2026Moving Beyond Diversity: Visual Token Pruning as Subspace Reconstruction for Efficient VLMs17 Jun
- 2026RetentiveKV: State-Space Memory for Uncertainty-Aware Multimodal KV Cache Eviction14 Apr
- 2025See What You Are Told: Visual Attention Sink in Large Multimodal Models5 Mar
- 2024Feather the Throttle: Revisiting Visual Token Pruning for Vision-Language Model Acceleration17 Dec
- 2024VL-Cache: Sparsity and Modality-Aware KV Cache Compression for Vision-Language Model Inference Acceleration29 Oct
- 2024An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models11 Mar
Other references · 6
- 2025KVzip: Query-Agnostic KV Cache Compression with Context Reconstructioncited by 89
- 2026Understanding the Physics of Key-Value Cache Compression for LLMs through Attention Dynamicscited by 5
- 2026A Simple Plug-in for Improving Eviction-Based KV Cache Compressioncited by 3
- 2025Causality neq Decodability, and Vice Versa: Lessons from Interpreting Counting ViTscited by 2
- 2026Dissociating Decodability and Causal Use in Bracket-Sequence Transformerscited by 2
- 2026The Vision Encoder as a Privacy Boundary: Visual-Token Side Channels in Encoder-Free Vision-Language Modelscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026LASER: A Corrective Lens for LVLMs via Visual Attention Preservation and Sink Suppression
- 2026Diagnosing Visual Ignorance in Vision-Language Models
- 2025Mixing Importance with Diversity: Joint Optimization for KV Cache Compression in Large Vision-Language Models
- 2026Seeing without Looking: Do Vision-Language Benchmarks Really Test Vision?
- 2024Efficient Inference of Vision Instruction-Following Models with Elastic Cache
- 2026Text Capability Loss in Vision-Language Adaptation: An Attention-Sink Diagnosis
- 2026Large Vision-Language Models Get Lost in Attention
- 2026Where To Look? : Causal Tracing of Vision Encoders in VLM