# GenPuzzle: Benchmarking Visual Reasoning in Image Generation Models

arXiv: https://arxiv.org/abs/2609.05902 · HTML: https://mlsanity.com/paper/2609.05902/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 18 of its 43 references that are in the map.
- First posted on arXiv 5 Sep 2026.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)

## Builds on (most cited first, 18 of 18)

- [Video models are zero-shot learners and reasoners](https://mlsanity.com/paper/2509.20328/) (2025)
- [EasyControl: Adding Efficient and Flexible Control for Diffusion Transformer](https://mlsanity.com/paper/2503.07027/) (2025)
- [MagicStick: Controllable Video Editing via Control Handle Transformations](https://mlsanity.com/paper/2312.03047/) (2023)
- [ZeroBench: An Impossible Visual Benchmark for Contemporary Large Multimodal Models](https://mlsanity.com/paper/2502.09696/) (2025)
- [BabyVision: Visual Reasoning Beyond Language](https://mlsanity.com/paper/2601.06521/) (2026)
- [EEdit: Rethinking the Spatial and Temporal Redundancy for Efficient Image Editing](https://mlsanity.com/paper/2503.10270/) (2025)
- [LayerTracer: Cognitive-Aligned Layered SVG Synthesis via Diffusion Transformer](https://mlsanity.com/paper/2502.01105/) (2025)
- [FonTS: Text Rendering with Typography and Style Controls](https://mlsanity.com/paper/2412.00136/) (2024)
- [InstantSwap: Fast Customized Concept Swapping across Sharp Shape Differences](https://mlsanity.com/paper/2412.01197/) (2024)
- [OmniConsistency: Learning Style-Agnostic Consistency from Paired Stylization Data](https://mlsanity.com/paper/2505.18445/) (2025)
- [ReasonGen-R1: CoT for Autoregressive Image generation models through SFT and RL](https://mlsanity.com/paper/2505.24875/) (2025)
- [EasyText: Controllable Diffusion Transformer for Multilingual Text Rendering](https://mlsanity.com/paper/2505.24417/) (2025)
- [RelationAdapter: Learning and Transferring Visual Relation with Diffusion Transformers](https://mlsanity.com/paper/2506.02528/) (2025)
- [DiffSim: Taming Diffusion Models for Evaluating Visual Similarity](https://mlsanity.com/paper/2412.14580/) (2024)
- [WordCon: Word-level Typography Control in Scene Text Rendering](https://mlsanity.com/paper/2506.21276/) (2025)
- [ReasonPix2Pix: Instruction Reasoning Dataset for Advanced Image Editing](https://mlsanity.com/paper/2405.11190/) (2024)
- [Reasoning to Edit: Hypothetical Instruction-Based Image Editing with Visual Reasoning](https://mlsanity.com/paper/2507.01908/) (2025)
- [Grid: Omni Visual Generation](https://mlsanity.com/paper/2412.10718/) (2024)

## Built on by (most cited first, 1 of 1)

- [Visual Abstention in Unified Multimodal Models](https://mlsanity.com/paper/2610.07887/) (2026)

## Abstract (arXiv)

Recent image generation systems increasingly combine multimodal understanding, reasoning, and synthesis, suggesting that they may do more than render plausible scenes. Yet existing evaluations emphasize aesthetics, prompt alignment, compositionality, or text-based answers, leaving unclear whether these systems can solve visual problems and faithfully express solutions in pixels. We introduce GenPuzzle, a benchmark for reasoning-centric image generation. GenPuzzle contains 2,005 problems across 12 tracks, spanning pattern completion, spatial construction, mazes, Sudoku, nonograms, tangrams, board games, matchstick puzzles, orthographic projection, and mathematical visual proof. Each task provides a visual puzzle and requires an image output that preserves the input state while executing a logically valid solution. GenPuzzle uses task-specific evaluation protocols: discrete grid outputs are transcribed and verified programmatically, while visually complex outputs are assessed with tiered, multidimensional, or binary multimodal large language model (MLLM) rubrics. We further select the automatic judge by measuring agreement with human reference scores. Across three frontier generators, the strongest model reaches only 40.57 Macro Overall, revealing frequent failures in logic, geometry, state preservation, and instruction execution. GenPuzzle provides a testbed for measuring progress from image rendering toward visual problem solving.