mlsanityData to 7 Oct 2026

Readable Yet Unpredictable: Rotated-Outcome Prediction in Vision-Language Models

arXiv 2606.07641 ↗First posted 1 Jun 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Can vision-language models predict what a 180deg rotation would reveal from the original image alone? We study this ability through Rotated-Outcome Prediction: given an original image, a model must answer what would be seen or read after a 180deg in-plane rotation, without directly observing the rotated target. To isolate this gap, we introduce RotOutBench, a paired diagnostic benchmark spanning open visual cases and controlled text-image rotations. A sharp pattern emerges: many VLMs can recognize the relevant content when directly given either the original or rotated image, yet fail to infer the rotated result from the original image alone. On controlled text-image rotations, predicted-rotation accuracy collapses to near zero even for models with high direct-reading accuracy. A model-level case study further shows that the prediction state can approach a rotated-image reading state, while the final readout still shifts toward the original string. Current VLMs can recognize a transformed visual state when it is shown, but often fail to predict that state from the original view.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 11 of its 29 references that are in the map. First posted on arXiv 1 Jun 2026.

Builds on

11 of 29 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Grounding Visual Illusions in Language: Do Vision-Language Models Perceive Illusions Like Humans?31 Oct

Other references · 10

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  3. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  4. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  5. 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 443
  6. 2023What's "up" with vision-language models? Investigating their struggle with spatial reasoningcited by 245
  7. 2024OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoningcited by 165
  8. 2025Perspective-Aware Reasoning in Vision-Language Models via Mental Imagery Simulationcited by 41
  9. 2024Is 'Right' Right? Enhancing Object Orientation Understanding in Multimodal Large Language Models through Egocentric Instruction Tuningcited by 7
  10. 2025RotBench: Evaluating Multimodal Large Language Models on Identifying Image Rotationcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Rotated, but How Far? Diagnosing and Improving Object-Rotation Reasoning in VLMs
  2. 2026Seeing without Looking: Do Vision-Language Benchmarks Really Test Vision?
  3. 2026Looks the Same, Answers Differently: Flip-Direction Steering for Robust Vision-Language Reasoning
  4. 2025Test-Time Consistency in Vision Language Models
  5. 2026Why MLLMs Struggle to Determine Object Orientations
  6. 2025TDBench: A Benchmark for Top-Down Image Understanding with Reliability Analysis of Vision-Language Models
  7. 2025SpinBench: Perspective and Rotation as a Lens on Spatial Reasoning in VLMs
  8. 2023Synthesize, Diagnose, and Optimize: Towards Fine-Grained Vision-Language Understanding