mlsanityData to 7 Oct 2026

JigShape: Evaluating Visual-Geometric Reasoning in VLMs through Jigsaw Puzzles

arXiv 2607.27670 ↗First posted 30 Jul 2026cs.CV cs.AIcited by 1

On the map: LLM reasoning → Spatial and embodied reasoning

Abstract (arXiv)

Jigsaw puzzle solving requires jointly reasoning about visual content and geometric constraints, yet existing benchmarks use rectangular cuts that create ambiguous ground truth in texture-repeated regions. We introduce textitours, a benchmark with tab-and-blank interlocking pieces where geometric constraints provide strong local compatibility requirements that, combined with visual content, yield unambiguous ground truth. Across 95K instances at four grid densities (4×4 to 16×16), we find that zero-shot VLMs largely lack geometric reasoning: only one of five frontier models (GPT-5.5) exceeds random baseline on 4×4 puzzles, while all others perform at chance level. While supervised fine-tuning achieves >97% on 4×4, all models collapse on larger grids: GPT-5.5 drops from 70% to near-random on 8×8, and even fine-tuned models fall below 5% on 12×12. This ``scaling cliff'' suggests current architectures cannot maintain consistent constraint satisfaction as the number of pieces increases. ours establishes scalable geometric reasoning as an open challenge for vision-language models.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 36 of its 45 references that are in the map. First posted on arXiv 30 Jul 2026.

Builds on

36 of 45 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Jigsaw-Puzzles: From Seeing to Understanding to Reasoning in Vision-Language Models27 May
  2. 2025LEGO-Puzzles: How Good Are MLLMs at Multi-Step Spatial Reasoning?25 Mar
  3. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities22 Jan
  4. 2023What's "up" with vision-language models? Investigating their struggle with spatial reasoning30 Oct
  5. 2022Visual Spatial Reasoning30 Apr

Other references · 31

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2023Visual Instruction Tuningcited by 7,792
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  5. 2025Qwen2.5-VL Technical Reportcited by 5,418
  6. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  7. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  8. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  9. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  10. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 529
  11. 2022Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionalitycited by 523
  12. 2019Domain Generalization by Solving Jigsaw Puzzlescited by 449
19 more
  1. 2021VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomenacited by 134
  2. 2023Panoptic Scene Graph Generation with Semantics-Prototype Learningcited by 42
  3. 2025Spatial-SSRL: Enhancing Spatial Understanding via Self-Supervised Reinforcement Learningcited by 38
  4. 2025Jigsaw-R1: A Study of Rule-based Visual Reinforcement Learning with Jigsaw Puzzlescited by 36
  5. 2022Masked Jigsaw Puzzle: A Versatile Position Embedding for Vision Transformerscited by 33
  6. 2025Treble Counterfactual VLMs: A Causal Approach to Hallucinationcited by 33
  7. 2024DiffAssemble: A Unified Graph-Diffusion Model for 2D and 3D Reassemblycited by 31
  8. 2024DPU: Dynamic Prototype Updating for Multimodal Out-of-Distribution Detectioncited by 28
  9. 2025VGRP-Bench: Visual Grid Reasoning Puzzle Benchmark for Large Vision-Language Modelscited by 26
  10. 2025Visual Jigsaw Post-Training Improves MLLMscited by 24
  11. 2022Jigsaw-ViT: Learning Jigsaw Puzzles in Vision Transformercited by 21
  12. 2025Secure On-Device Video OOD Detection Without Backpropagationcited by 19
  13. 2017A Genetic Algorithm-Based Solver for Very Large Jigsaw Puzzlescited by 12
  14. 2026FORTIS: Benchmarking Over-Privilege in Agent Skillscited by 11
  15. 2026Defenses Against Prompt Attacks Learn Surface Heuristicscited by 10
  16. 2025PuzzleCraft: Exploration-Aware Curriculum Learning for Puzzle-Based RLVR in VLMscited by 4
  17. 2025VLHSA: Vision-Language Hierarchical Semantic Alignment for Jigsaw Puzzle Solving with Eroded Gapscited by 3
  18. 2025PuzLM: Solving Jigsaw Puzzles with Sequence-to-Sequence Language Modelscited by 3
  19. 2026Geometry over Density: Few-Shot Cross-Domain OOD Detectioncited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026GTA: Graph Theory Agent and Benchmark for Algorithmic Graph Reasoning with LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Reasoning or Pattern Matching? Probing Large Vision-Language Models with Visual Puzzles
  2. 2023GeomVerse: A Systematic Evaluation of Large Models for Geometric Reasoning
  3. 2026TACIT Benchmark: A Programmatic Visual Reasoning Benchmark for Generative and Discriminative Models
  4. 2024EAGLE: Elevating Geometric Reasoning through LLM-empowered Visual Instruction Tuning
  5. 2026VRIQ: Benchmarking and Analyzing Visual-Reasoning IQ of VLMs
  6. 2025GeoBench: Rethinking Multimodal Geometric Problem-Solving via Hierarchical Evaluation
  7. 2022Abstract Visual Reasoning with Tangram Shapes
  8. 2026Make Geometry Matter for Spatial Reasoning