FLIP: Final Layer Inference-Time Probing for Vision-Language Models
On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning also fits: Embodied and 3D vision-language models
Abstract (arXiv)
We present FLIP, a final-layer inference-time probe for testing whether a logit-facing intervention site in an open-weight vision-language model (VLM) supports structured, task-linked computation rather than generic perturbation. Behavioral change under internal intervention is otherwise mechanistically ambiguous: it may reflect improved use of visual evidence, generic output instability, or outright degradation. FLIP applies elementwise flooring to the final normalized hidden state before logit computation, leaving parameters, prompts, and decoding unchanged. On a controlled detection/counting probe, sweeping intervention strength reveals three regions: negligible change, a bounded interior regime in which detection recall at IoU 0.50 (R50) improves while tolerant counting error (Ecount) falls, and over-suppression. We formalize a four-criterion probe-and-sweep protocol for disciplining the interpretation of intervention effects: regime structure, grounding-proxy alignment, feature-coherence dependence, and failure to reproduce the same positive regime on a performance-based negative control. The post-normalization state passed to the output head is the logit-facing instantiation of this test; under a non-targeted flooring sweep it satisfies the full protocol. Raw decoder-layer interventions, including the last-block output before final normalization, and the singleton-pair left/right control fail to reproduce the Final-site signature, while same-site operators and multiple VLMs replicate it. FLIP is therefore a validation step for intervention-based mechanistic interpretability, not a steering method. Repo at https://github.com/earl-juanico/flip-qwen3vl/
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 10 of its 14 references that are in the map. First posted on arXiv 25 Sep 2026.
Builds on
10 of 14 references.
Closest ancestors · 3
Same direction, newest first
- 2026Circuit Tracing in Vision-Language Models: Understanding the Internal Mechanisms of Multimodal Thinking23 Feb
- 2025Sparse Autoencoders Learn Monosemantic Features in Vision-Language Models3 Apr
- 2024Towards Interpreting Visual Information Processing in Vision-Language Models9 Oct
Other references · 7
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
- 2021Causal Abstractions of Neural Networkscited by 401
- 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 328
- 2025The Hidden Life of Tokens: Reducing Hallucination of Large Vision-Language Models via Visual Information Steeringcited by 62
- 2025Interpretable Open-Vocabulary Referring Object Detection with Reverse Contrast Attentioncited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Limits and Gains of Test-Time Scaling in Vision-Language Reasoning
- 2026Token-Disentangled Latent Test-Time Scaling for Vision-Language Reasoning
- 2026Seeing without Looking: Do Vision-Language Benchmarks Really Test Vision?
- 2026On Test-Time Scaling for Vision-Language Models
- 2026Not All Error Yields to Scale: Where Scaling Stops in Vision-Language Inference
- 2026A Cross-Architecture Audit of Direction-Based Inference-Time Defences in Vision-Language Models
- 2026Where Reliability Lives in Vision-Language Models: A Mechanistic Study of Attention, Hidden States, and Causal Circuits
- 2026Dismantling Pathological Shortcuts: A Causal Framework for Faithful LVLM Decoding