mlsanityData to 14 Oct 2026

Readout is not Recovery: Dissociating Coordinate Emission from Visual-Corruption Repair in Vision-Language Models

arXiv 2609.32757 ↗First posted 26 Sep 2026cs.AI cs.CV cs.LGcited by 0

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

Abstract (arXiv)

VLM bounding-box localization is both language generation and spatial commitment. Parseable fields such as bbox2d make localization easy to score, but dimensions that emit coordinate tokens need not repair localization after visual evidence is damaged. We study this readout/recovery separation in Qwen3-VL-4B-Instruct on single-object COCO grounding. We compare clean coordinate-token readout rankings with corruption-derived repair rankings, using object-mask endpoint replacement for recovery and clean-input flooring for depth localization. In Qwen3-VL, coordinate-token rankings are inert through layer 24, load-bearing from layers 32-35, and peak at layer 34; corruption-derived rankings harm layers 16-24 but become beneficial near layer 35/final. A Kimi-VL-A3B diagnostic shows a matching output-proximal transition despite a different box format. Object-mask recovery separates rank budgets: k=250 shows necessity, k=500 shows Top-k restoration above random, and k=d/2 is largely capacity-driven. Partial-occlusion sweeps reveal that high-overlap coordinate-token sets can hurt at k=1000 and help mainly at half-width, while population corruption-derived sets provide no reliable fixed repair set. Edge-attribution patching shows coordinate-token paths are high precision but low recall for detection recovery, and RMSNorm quasi-layer controls do not close the endpoint-repair gap. Endpoint coordinate triage is therefore a useful circuit prior, but occlusion recovery requires a separate benchmark.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 18 of its 21 references that are in the map. First posted on arXiv 26 Sep 2026.

Builds on

18 of 21 references.

References · 18

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2016Layer Normalizationcited by 5,669
  3. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  4. 2025Qwen3-VL Technical Reportcited by 2,793
  5. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  6. 2019Root Mean Square Layer Normalizationcited by 1,480
  7. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  8. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
  9. 2019Attention is not Explanationcited by 929
  10. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
  11. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 724
  12. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 599
6 more
  1. 2023Ferret: Refer and Ground Anything Anywhere at Any Granularitycited by 509
  2. 2019Is Attention Interpretable?cited by 389
  3. 2025Kimi-VL Technical Reportcited by 388
  4. 2021Pix2seq: A Language Modeling Framework for Object Detectioncited by 349
  5. 2019Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Comingcited by 336
  6. 2020Robust Object Detection under Occlusion with Context-Aware CompositionalNetscited by 54

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding
  2. 2026MissingBench-Verified: Probing Vision-Language Models' Inability to Detect Missing Object Parts
  3. 2025Mitigating Coordinate Prediction Bias from Positional Encoding Failures
  4. 2025Pathological Truth Bias in Vision-Language Models
  5. 2026Edge Reliability Gap in Vision-Language Models: Quantifying Failure Modes of Compressed VLMs Under Visual Corruption
  6. 2026Reroute, Don't Remove: Recoverable Visual Token Routing for Vision-Language Models
  7. 2025Grounding-Aware Token Pruning: Recovering from Drastic Performance Drops in Visual Grounding Caused by Pruning
  8. 2026From Patching to Pruning Visual Computation in Vision Language Models