Image-Space Rule Discovery
Not on the map. Why?
Abstract (arXiv)
Can image-editing models discover visual rules in image space and complete problem-solving end-to-end? We tackle this question in the spirit of a human worksheet test (e.g., an IQ test), using problems that require models to read image-based instructions, recognize the problem, infer the answer, bind it to the correct destination, control output count, suppress unnecessary edits, and preserve the input and format. We introduce WISRD, a Worksheet Image-Space Rule Discovery benchmark with 11 core tasks under eight information conditions, spanning localized marking, filling, copying, counting, and no-edit suppression, together with four supplementary reasoning-stress probes for multi-step spatial manipulation, abstract pattern reasoning, logical inference, and constraint-based problem solving. We identify three key findings as follows. (i) Among the frontier image-editing models evaluated, Nano Banana Pro achieves the highest score. On the shared V0--V3 no-reference subset, the Auto-Strict proxy pass rates are 48.7% for Nano Banana Pro, 13.4% for Qwen-Image-Edit, 11.5% for FLUX.2 Klein 4B API, 11.3% for FLUX.2 Klein 4B open-weight, and 0.0% for InstructPix2Pix. (ii) Analysis reveals that current image-editing models can partially rely on rendered in-image instructions even when the external prompt is absent or merely generic. (iii) In small supplementary diagnostics, Nano Banana Pro achieves 70.0% on 4-by-4 Sudoku and 22.9% on public RAVEN pattern-discovery items in image space.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 34 of its 39 references that are in the map. First posted on arXiv 1 Aug 2026.
Builds on
34 of 39 references.
References · 34
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
- 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
- 2023Visual Instruction Tuningcited by 7,792
- 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
- 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
22 more
- 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
- 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
- 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
- 2025Qwen-Image Technical Reportcited by 973
- 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
- 2024Emu3: Next-Token Prediction is All You Needcited by 761
- 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 615
- 2023TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answeringcited by 416
- 2023T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generationcited by 253
- 2019RAVEN: A Dataset for Relational and Analogical Visual rEasoNingcited by 225
- 2025Video models are zero-shot learners and reasonerscited by 208
- 2024Vision language models are blind: Failing to translate detailed visual features into wordscited by 144
- 2025Envisioning Beyond the Pixels: Benchmarking Reasoning-Informed Visual Editingcited by 97
- 2025KRIS-Bench: Benchmarking Next-Level Intelligent Image Editing Modelscited by 69
- 2025Vision Language Models are Biasedcited by 63
- 2026Image Generators are Generalist Vision Learnerscited by 40
- 2025UniREditBench: A Unified Reasoning-based Image Editing Benchmarkcited by 23
- 2025Is Nano Banana Pro a Low-Level Vision All-Rounder? A Comprehensive Evaluation on 14 Tasks and 40 Datasetscited by 19
- 2025GIR-Bench: Versatile Benchmark for Generating Images with Reasoningcited by 15
- 2026Show, Don't Tell: Evaluating Spatial Cognition in Generative Pixels Rather Than LLM Textcited by 2
- 2024Are VLMs Really Blindcited by 1
- 2026Image Editing Models are Numerical Solverscited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.