mlsanityData to 7 Oct 2026

Image-Space Rule Discovery

arXiv 2608.00490 ↗First posted 1 Aug 2026cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

Can image-editing models discover visual rules in image space and complete problem-solving end-to-end? We tackle this question in the spirit of a human worksheet test (e.g., an IQ test), using problems that require models to read image-based instructions, recognize the problem, infer the answer, bind it to the correct destination, control output count, suppress unnecessary edits, and preserve the input and format. We introduce WISRD, a Worksheet Image-Space Rule Discovery benchmark with 11 core tasks under eight information conditions, spanning localized marking, filling, copying, counting, and no-edit suppression, together with four supplementary reasoning-stress probes for multi-step spatial manipulation, abstract pattern reasoning, logical inference, and constraint-based problem solving. We identify three key findings as follows. (i) Among the frontier image-editing models evaluated, Nano Banana Pro achieves the highest score. On the shared V0--V3 no-reference subset, the Auto-Strict proxy pass rates are 48.7% for Nano Banana Pro, 13.4% for Qwen-Image-Edit, 11.5% for FLUX.2 Klein 4B API, 11.3% for FLUX.2 Klein 4B open-weight, and 0.0% for InstructPix2Pix. (ii) Analysis reveals that current image-editing models can partially rely on rendered in-image instructions even when the external prompt is absent or merely generic. (iii) In small supplementary diagnostics, Nano Banana Pro achieves 70.0% on 4-by-4 Sudoku and 22.9% on public RAVEN pattern-discovery items in image space.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 34 of its 39 references that are in the map. First posted on arXiv 1 Aug 2026.

Builds on

34 of 39 references.

References · 34

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  6. 2023Visual Instruction Tuningcited by 7,792
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  8. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  9. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  10. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  11. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  12. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
22 more
  1. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  2. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  3. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
  4. 2025Qwen-Image Technical Reportcited by 973
  5. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
  6. 2024Emu3: Next-Token Prediction is All You Needcited by 761
  7. 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 615
  8. 2023TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answeringcited by 416
  9. 2023T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generationcited by 253
  10. 2019RAVEN: A Dataset for Relational and Analogical Visual rEasoNingcited by 225
  11. 2025Video models are zero-shot learners and reasonerscited by 208
  12. 2024Vision language models are blind: Failing to translate detailed visual features into wordscited by 144
  13. 2025Envisioning Beyond the Pixels: Benchmarking Reasoning-Informed Visual Editingcited by 97
  14. 2025KRIS-Bench: Benchmarking Next-Level Intelligent Image Editing Modelscited by 69
  15. 2025Vision Language Models are Biasedcited by 63
  16. 2026Image Generators are Generalist Vision Learnerscited by 40
  17. 2025UniREditBench: A Unified Reasoning-based Image Editing Benchmarkcited by 23
  18. 2025Is Nano Banana Pro a Low-Level Vision All-Rounder? A Comprehensive Evaluation on 14 Tasks and 40 Datasetscited by 19
  19. 2025GIR-Bench: Versatile Benchmark for Generating Images with Reasoningcited by 15
  20. 2026Show, Don't Tell: Evaluating Spatial Cognition in Generative Pixels Rather Than LLM Textcited by 2
  21. 2024Are VLMs Really Blindcited by 1
  22. 2026Image Editing Models are Numerical Solverscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.