mlsanityData to 14 Oct 2026

StateSight: Benchmarking Latent Spatial-State Reconstruction in Vision-Language Models

arXiv 2608.20414 ↗First posted 15 Aug 2026cs.AI cs.CVcited by 1

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Vision-language models are increasingly used for multimodal question answering, yet their ability to reconstruct latent spatial structure from a single image remains difficult to isolate. Broad benchmarks often combine perception, optical character recognition, domain knowledge, linguistic priors, and reasoning in the same evaluation. We introduce StateSight, a procedurally generated benchmark for cube-net opposite-face reasoning, occluded cube-tower counting, and 4-neighbor connected-component counting. Each task family contains 300 single-image prompts with deterministic oracle labels and exact-match scoring. OpenAI GPT-5.5, using the API model identifier gpt-5.5, achieved 59.3%, 33.3%, and 28.3% accuracy across the three tasks, while Claude Sonnet 5 achieved 53.3%, 18.7%, and 7.3%. All final direct runs had zero format errors. A 30-participant human baseline on 60 items exceeded both models on every task, with mean accuracies of 80.8%, 68.8%, and 64.3%. Visible-derivation analysis identified recurring errors in image-state reconstruction and reasoning procedure. We also introduce StateSight-Steps, a companion dataset of 900 interleaved image-text examples and 3,600 deterministic intermediate visual states. The results show that format-valid responses can mask failures to recover the spatial structure required for verifiable visual inference.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 9 of its 13 references that are in the map. First posted on arXiv 15 Aug 2026.

Builds on

9 of 13 references.

References · 9

  1. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  2. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
  3. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 550
  4. 2022Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionalitycited by 529
  5. 2023Perception Test: A Diagnostic Benchmark for Multimodal Video Modelscited by 314
  6. 2023What's "up" with vision-language models? Investigating their struggle with spatial reasoningcited by 254
  7. 2019SpatialSense: An Adversarially Crowdsourced Benchmark for Spatial Relation Recognitioncited by 55
  8. 2025When Visualizing is the First Step to Reasoning: MIRA, a Benchmark for Visual Chain-of-Thoughtcited by 21
  9. 2026Visual Generation Unlocks Human-Like Reasoning through Multimodal World Modelscited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026HumanPhys: A Human-Centric Physical-World Perception Benchmark for Real-World Visual Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Cube Bench: A Benchmark for Spatial Visual Reasoning in MLLMs
  2. 2026Retrieve, Integrate, and Synthesize: Spatial-Semantic Grounded Latent Visual Reasoning
  3. 2026Do Vision-Language Models See or Guess? Measuring and Reducing Textual-Prior Reliance with a Phrasing-Controlled Benchmark
  4. 2025Beyond the Visible: Benchmarking Occlusion Perception in Multimodal Large Language Models
  5. 2026From Perception to Integration: Revisiting the Internal Dynamics of Reasoning in Vision-Language Models
  6. 2026Seeing is not Enough: Vision-Language Models Perceive Evidence but Fail to Act
  7. 2026Geometric Parrots: Rule Absence vs. Rule Fragility in Spatial and Physical Reasoning for Vision-Language Models
  8. 2026VISTAQA: Benchmarking Joint Visual Question Answering and Pixel-Level Evidence