# Conditional Visual Evidence Utility: State-Dependent Rank Reversals in Frozen Vision-Language Encoders

arXiv: https://arxiv.org/abs/2608.28316 · HTML: https://mlsanity.com/paper/2608.28316/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 10 references that are in the map.
- First posted on arXiv 28 Aug 2026.

## Builds on (most cited first, 10 of 10)

- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models](https://mlsanity.com/paper/2403.06764/) (2024)
- [V*: Guided Visual Search as a Core Mechanism in Multimodal LLMs](https://mlsanity.com/paper/2312.14135/) (2023)
- [Token Pruning in Multimodal Large Language Models: Are We Solving the Right Problem?](https://mlsanity.com/paper/2502.11501/) (2025)
- [Learning to Maximize Mutual Information for Dynamic Feature Selection](https://mlsanity.com/paper/2301.00557/) (2023)
- [Active Video Perception: Iterative Evidence Seeking for Agentic Long Video Understanding](https://mlsanity.com/paper/2512.05774/) (2025)
- [Estimating Conditional Mutual Information for Dynamic Feature Selection](https://mlsanity.com/paper/2306.03301/) (2023)
- [Rethinking Visual Token Reduction in LVLMs Under Cross-Modal Misalignment](https://mlsanity.com/paper/2506.22283/) (2025)
- [Question-guided Visual Compression with Memory Feedback for Long-Term Video Understanding](https://mlsanity.com/paper/2603.15167/) (2026)
- [The Perceptual Bandwidth Bottleneck in Vision-Language Models: Active Visual Reasoning via Sequential Experimental Design](https://mlsanity.com/paper/2605.01345/) (2026)

## Abstract (arXiv)

Static importance scores compress visual evidence into a single ranking, but the value of remaining evidence can change after one cue has been observed. We study this possibility in controlled compositional visual search, where color, shape, and texture evidence can be independently exposed and their conditional marginal utility measured across acquisition states. In a held-out confirmation on 800 scenes, frozen OpenCLIP and SigLIP exhibit robust state-dependent rank reversals that concentrate in candidate-overlap regimes designed to induce ordering changes, persist across two evidence-accumulation constructions and ten equivalent query wordings, and collapse to near-chance-scale behavior under query-scene derangement. A subsequent role-balanced follow-up on 1,200 scenes rotates the abstract roles of initially strong, redundancy-inducing, and comparator attributes; the positive-minus-negative reversal contrast remains positive across all 24 role-permutation, backbone, and evidence-mode cells, although residual attribute-identity effects remain. We further distinguish measured replanning opportunity from prospective predictability. Matched-first-action utility analyses show substantial opportunity to rerank remaining evidence, but lightweight predictors using posterior-based or acquired-embedding state representations do not establish a robust incremental advantage of acquired-state information over legal static controls on the role-balanced benchmark. Together, these results show that conditional visual evidence utility is reliably state dependent in this controlled setting, while separating the existence of changing utility from the stronger claim that those changes are prospectively predictable by a learned selector.