ReasonPix2Pix: Instruction Reasoning Dataset for Advanced Image Editing
Not on the map. Why?
Abstract (arXiv)
Instruction-based image editing focuses on equipping a generative model with the capacity to adhere to human-written instructions for editing images. Current approaches typically comprehend explicit and specific instructions. However, they often exhibit a deficiency in executing active reasoning capacities required to comprehend instructions that are implicit or insufficiently defined. To enhance active reasoning capabilities and impart intelligence to the editing model, we introduce ReasonPix2Pix, a comprehensive reasoning-attentive instruction editing dataset. The dataset is characterized by 1) reasoning instruction, 2) more realistic images from fine-grained categories, and 3) increased variances between input and edited images. When fine-tuned with our dataset under supervised conditions, the model demonstrates superior performance in instructional editing tasks, independent of whether the tasks require reasoning or not. The code will be available at https://github.com/Jin-Ying/ReasonPix2Pix.
Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 58 of its 67 references that are in the map. First posted on arXiv 18 May 2024.
Builds on
58 of 67 references.
References · 58
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
- 2023Visual Instruction Tuningcited by 7,869
- 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
- 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
- 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
- 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
38 more
- 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
- 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
- 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
- 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
- 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
- 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
- 2021Image Super-Resolution via Iterative Refinementcited by 1,511
- 2021Palette: Image-to-Image Diffusion Modelscited by 1,300
- 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,190
- 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,179
- 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,178
- 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,163
- 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
- 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
- 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 991
- 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 927
- 2018Multimodal Unsupervised Image-to-Image Translationcited by 795
- 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
- 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 668
- 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulationcited by 642
- 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 624
- 2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 550
- 2021Designing an Encoder for StyleGAN Image Manipulationcited by 544
- 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Actioncited by 517
- 2023VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Taskscited by 501
- 2015A Neural Algorithm of Artistic Stylecited by 483
- 2019Image2StyleGAN++: How to Edit the Embedded Images?cited by 356
- 2022Text2LIVE: Text-Driven Layered Image and Video Editingcited by 312
- 2023GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interestcited by 302
- 2022VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidancecited by 285
- 2022EDICT: Exact Diffusion Inversion via Coupled Transformationscited by 258
- 2023GPT4Tools: Teaching Large Language Model to Use Tools via Self-instructioncited by 248
- 2019Few-Shot Unsupervised Image-to-Image Translationcited by 222
- 2021HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editingcited by 208
- 2021Few-shot Image Generation via Cross-domain Correspondencecited by 207
- 2021CLIPstyler: Image Style Transfer with a Single Text Conditioncited by 203
50 most cited of 58 All 58 (JSON)
Built on by
Citing papers per year: 2024: 2; 2025: 8; 2026: 4.
14 papers.
Sort:
By area:
- 2025Reflect-DiT: Inference-Time Scaling for Text-to-Image Diffusion Transformers via In-Context Reflectioncited by 43
- 2024EditWorld: Simulating World Dynamics for Instruction-Following Image Editingcited by 38
- 2024Trans4D: Realistic Geometry-Aware Transition for Compositional Text-to-4D Synthesiscited by 19
- 2025VINCIE: Unlocking In-context Image Editing from Videocited by 17
- 2025PICABench: How Far Are We from Physically Realistic Image Editing?cited by 16
- 2025ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal Generationcited by 13
- 2025Reasoning to Edit: Hypothetical Instruction-Based Image Editing with Visual Reasoningcited by 9
- 2026MiLDEdit: Reasoning-Based Multi-Layer Design Document Editingcited by 3
- 2026WorldEdit: Towards Open-World Image Editing with a Knowledge-Informed Benchmarkcited by 3
- 2025FireScope: Wildfire Risk Raster Prediction with a Chain-of-Thought Oraclecited by 2
- 2025UmniBench: Unified Understand and Generation Model Oriented Omni-dimensional Benchmarkcited by 2
- 2026GenPuzzle: Benchmarking Visual Reasoning in Image Generation Modelscited by 1
- 2026GenPuzzle: Benchmarking Visual Reasoning in Image Generation Models5 Sep 2026
- 2026Instruction-based Image Editing: A Survey on Data, Models, Evaluation, and Applications28 Jul 2026
- 2026WorldEdit: Towards Open-World Image Editing with a Knowledge-Informed Benchmark6 Feb 2026
- 2026MiLDEdit: Reasoning-Based Multi-Layer Design Document Editing8 Jan 2026
- 2025UmniBench: Unified Understand and Generation Model Oriented Omni-dimensional Benchmark19 Dec 2025
- 2025FireScope: Wildfire Risk Raster Prediction with a Chain-of-Thought Oracle21 Nov 2025
- 2025ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal Generation3 Nov 2025
- 2025Understanding the Implicit User Intention via Reasoning with Large Language Model for Image Editing31 Oct 2025
- 2025PICABench: How Far Are We from Physically Realistic Image Editing?20 Oct 2025
- 2025Reasoning to Edit: Hypothetical Instruction-Based Image Editing with Visual Reasoning2 Jul 2025
- 2025VINCIE: Unlocking In-context Image Editing from Video12 Jun 2025
- 2025Reflect-DiT: Inference-Time Scaling for Text-to-Image Diffusion Transformers via In-Context Reflection15 Mar 2025
- 2025Reflect-DiT: Inference-Time Scaling for Text-to-Image Diffusion Transformers via In-Context Reflectioncited by 43
- 2025Reasoning to Edit: Hypothetical Instruction-Based Image Editing with Visual Reasoningcited by 9
- 2025FireScope: Wildfire Risk Raster Prediction with a Chain-of-Thought Oraclecited by 2
- 2026GenPuzzle: Benchmarking Visual Reasoning in Image Generation Modelscited by 1
- 2025Understanding the Implicit User Intention via Reasoning with Large Language Model for Image Editingcited by 0
- 2026GenPuzzle: Benchmarking Visual Reasoning in Image Generation Models5 Sep 2026
- 2025FireScope: Wildfire Risk Raster Prediction with a Chain-of-Thought Oracle21 Nov 2025
- 2025Understanding the Implicit User Intention via Reasoning with Large Language Model for Image Editing31 Oct 2025
- 2025Reasoning to Edit: Hypothetical Instruction-Based Image Editing with Visual Reasoning2 Jul 2025
- 2025Reflect-DiT: Inference-Time Scaling for Text-to-Image Diffusion Transformers via In-Context Reflection15 Mar 2025
- 2025VINCIE: Unlocking In-context Image Editing from Videocited by 17
- 2025PICABench: How Far Are We from Physically Realistic Image Editing?cited by 16
- 2025ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal Generationcited by 13
- 2026MiLDEdit: Reasoning-Based Multi-Layer Design Document Editingcited by 3
- 2026WorldEdit: Towards Open-World Image Editing with a Knowledge-Informed Benchmarkcited by 3
- 2025UmniBench: Unified Understand and Generation Model Oriented Omni-dimensional Benchmarkcited by 2
- 2026Instruction-based Image Editing: A Survey on Data, Models, Evaluation, and Applicationscited by 0
- 2026Instruction-based Image Editing: A Survey on Data, Models, Evaluation, and Applications28 Jul 2026
- 2026WorldEdit: Towards Open-World Image Editing with a Knowledge-Informed Benchmark6 Feb 2026
- 2026MiLDEdit: Reasoning-Based Multi-Layer Design Document Editing8 Jan 2026
- 2025UmniBench: Unified Understand and Generation Model Oriented Omni-dimensional Benchmark19 Dec 2025
- 2025ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal Generation3 Nov 2025
- 2025PICABench: How Far Are We from Physically Realistic Image Editing?20 Oct 2025
- 2025VINCIE: Unlocking In-context Image Editing from Video12 Jun 2025
Similar, not cited
Close in wording, no citation either way
- 2023SmartEdit: Exploring Complex Instruction-based Image Editing with Multimodal Large Language Models
- 2025EditThinker: Unlocking Iterative Reasoning for Any Image Editor
- 2024InsightEdit: Towards Better Instruction Following for Image Editing
- 2025ReasonEdit: Towards Reasoning-Enhanced Image Editing Models
- 2026Is This Edit Correct? A Multi-Dimensional Benchmark for Reasoning-Aware Image Editing
- 2026MSRAMIE: Multimodal Structured Reasoning Agent for Multi-instruction Image Editing
- 2026Instruction-based Image Editing with Planning, Reasoning, and Generation
- 2026ThinkRL-Edit: Thinking in Reinforcement Learning for Reasoning-Centric Image Editing