# Pixel Decodability Is Not a Compression Signal: Causally Evaluating Importance Proxies for Visual KV-Cache Eviction

arXiv: https://arxiv.org/abs/2609.13012 · HTML: https://mlsanity.com/paper/2609.13012/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 12 of its 13 references that are in the map.
- First posted on arXiv 27 Jul 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Visual token pruning and efficiency](https://mlsanity.com/mllm/visual-token-pruning-and-efficiency/)
- Efficient LLM inference and serving / [Visual token pruning in multimodal models](https://mlsanity.com/inference/visual-token-pruning-in-multimodal-models/)

## Builds on (most cited first, 12 of 12)

- [An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models](https://mlsanity.com/paper/2403.06764/) (2024)
- [See What You Are Told: Visual Attention Sink in Large Multimodal Models](https://mlsanity.com/paper/2503.03321/) (2025)
- [KVzip: Query-Agnostic KV Cache Compression with Context Reconstruction](https://mlsanity.com/paper/2505.23416/) (2025)
- [VL-Cache: Sparsity and Modality-Aware KV Cache Compression for Vision-Language Model Inference Acceleration](https://mlsanity.com/paper/2410.23317/) (2024)
- [Feather the Throttle: Revisiting Visual Token Pruning for Vision-Language Model Acceleration](https://mlsanity.com/paper/2412.13180/) (2024)
- [Understanding the Physics of Key-Value Cache Compression for LLMs through Attention Dynamics](https://mlsanity.com/paper/2603.01426/) (2026)
- [A Simple Plug-in for Improving Eviction-Based KV Cache Compression](https://mlsanity.com/paper/2605.23258/) (2026)
- [RetentiveKV: State-Space Memory for Uncertainty-Aware Multimodal KV Cache Eviction](https://mlsanity.com/paper/2605.04075/) (2026)
- [Moving Beyond Diversity: Visual Token Pruning as Subspace Reconstruction for Efficient VLMs](https://mlsanity.com/paper/2606.18681/) (2026)
- [Causality neq Decodability, and Vice Versa: Lessons from Interpreting Counting ViTs](https://mlsanity.com/paper/2510.09794/) (2025)
- [Dissociating Decodability and Causal Use in Bracket-Sequence Transformers](https://mlsanity.com/paper/2604.22128/) (2026)
- [The Vision Encoder as a Privacy Boundary: Visual-Token Side Channels in Encoder-Free Vision-Language Models](https://mlsanity.com/paper/2606.14783/) (2026)

## Abstract (arXiv)

Vision-language models retain a substantial amount of pixel-decodable visual content in their visual key-value cache. We show, in our setting, that this retention is task-inert: across our preregistered tests, how much a unit retains never positively tracks whether the computation that answers the question causally relies on it. We measure retention with a learned pixel-inversion decoder and causal use with single-super-patch KV ablation, the teacher-forced drop in gold-answer log-probability, and relate the two within images under a preregistered, sign-calibrated, held-out design. Retention is decoupled from attention and, in a well-powered null, from causal utilization. Utilization is not inert to every proxy: attention weakly but significantly tracks it, the only signal we find that does and the design's positive control. We characterize pixel-decodable retention as an informational axis of the visual KV cache, orthogonal to the functional one. How much task-inert content a cache holds differs by architecture in our model pair: the encoder-free model retains 2.7 times more than the encoder-based one. The engineering consequence is a controlled negative result. At super-patch granularity, deconfounded pixel-decodable retention ranks KV eviction no better than random; at token granularity it acquires only a weak inverse-importance signal at larger budgets, dominated at every budget by attention magnitude. In our setting, pixel-decodable reconstructability is not a competitive KV-compression signal at any granularity we test.