mlsanityData to 14 Oct 2026

ReasonPix2Pix: Instruction Reasoning Dataset for Advanced Image Editing

arXiv 2405.11190 ↗First posted 18 May 2024cs.CVcited by 14

Not on the map. Why?

Abstract (arXiv)

Instruction-based image editing focuses on equipping a generative model with the capacity to adhere to human-written instructions for editing images. Current approaches typically comprehend explicit and specific instructions. However, they often exhibit a deficiency in executing active reasoning capacities required to comprehend instructions that are implicit or insufficiently defined. To enhance active reasoning capabilities and impart intelligence to the editing model, we introduce ReasonPix2Pix, a comprehensive reasoning-attentive instruction editing dataset. The dataset is characterized by 1) reasoning instruction, 2) more realistic images from fine-grained categories, and 3) increased variances between input and edited images. When fine-tuned with our dataset under supervised conditions, the model demonstrates superior performance in instructional editing tasks, independent of whether the tasks require reasoning or not. The code will be available at https://github.com/Jin-Ying/ReasonPix2Pix.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 58 of its 67 references that are in the map. First posted on arXiv 18 May 2024.

Builds on

58 of 67 references.

References · 58

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  8. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  9. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  10. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  11. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  12. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
38 more
  1. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  2. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  3. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  4. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  5. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  6. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  7. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
  8. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
  9. 2021Image Super-Resolution via Iterative Refinementcited by 1,511
  10. 2021Palette: Image-to-Image Diffusion Modelscited by 1,300
  11. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,190
  12. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,179
  13. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,178
  14. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,163
  15. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
  16. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
  17. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 991
  18. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 927
  19. 2018Multimodal Unsupervised Image-to-Image Translationcited by 795
  20. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  21. 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 668
  22. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulationcited by 642
  23. 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 624
  24. 2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 550
  25. 2021Designing an Encoder for StyleGAN Image Manipulationcited by 544
  26. 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Actioncited by 517
  27. 2023VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Taskscited by 501
  28. 2015A Neural Algorithm of Artistic Stylecited by 483
  29. 2019Image2StyleGAN++: How to Edit the Embedded Images?cited by 356
  30. 2022Text2LIVE: Text-Driven Layered Image and Video Editingcited by 312
  31. 2023GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interestcited by 302
  32. 2022VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidancecited by 285
  33. 2022EDICT: Exact Diffusion Inversion via Coupled Transformationscited by 258
  34. 2023GPT4Tools: Teaching Large Language Model to Use Tools via Self-instructioncited by 248
  35. 2019Few-Shot Unsupervised Image-to-Image Translationcited by 222
  36. 2021HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editingcited by 208
  37. 2021Few-shot Image Generation via Cross-domain Correspondencecited by 207
  38. 2021CLIPstyler: Image Style Transfer with a Single Text Conditioncited by 203

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 8; 2026: 4.

14 papers.

Sort:

By area:

Similar, not cited

Close in wording, no citation either way

  1. 2023SmartEdit: Exploring Complex Instruction-based Image Editing with Multimodal Large Language Models
  2. 2025EditThinker: Unlocking Iterative Reasoning for Any Image Editor
  3. 2024InsightEdit: Towards Better Instruction Following for Image Editing
  4. 2025ReasonEdit: Towards Reasoning-Enhanced Image Editing Models
  5. 2026Is This Edit Correct? A Multi-Dimensional Benchmark for Reasoning-Aware Image Editing
  6. 2026MSRAMIE: Multimodal Structured Reasoning Agent for Multi-instruction Image Editing
  7. 2026Instruction-based Image Editing with Planning, Reasoning, and Generation
  8. 2026ThinkRL-Edit: Thinking in Reinforcement Learning for Reasoning-Centric Image Editing