CogniEdit: Dense Gradient Flow Optimization for Fine-Grained Image Editing
On the map: Diffusion models → Image generation, text-to-image and editing
Abstract (arXiv)
Instruction-based image editing with diffusion models has achieved impressive results, yet existing methods struggle with fine-grained instructions specifying precise attributes such as colors, positions, and quantities. While recent approaches employ Group Relative Policy Optimization (GRPO) for alignment, they optimize only at individual sampling steps, providing sparse feedback that limits trajectory-level control. We propose a unified framework CogniEdit, combining multi-modal reasoning with dense reward optimization that propagates gradients across consecutive denoising steps, enabling trajectory-level gradient flow through the sampling process. Our method comprises three components: (1) Multi-modal Large Language Models for decomposing complex instructions into actionable directives, (2) Dynamic Token Focus Relocation that adaptively emphasizes fine-grained attributes, and (3) Dense GRPO-based optimization that propagates gradients across consecutive steps for trajectory-level supervision. Extensive experiments on benchmark datasets demonstrate that our CogniEdit achieves state-of-the-art performance in balancing fine-grained instruction following with visual quality and editability preservation
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 43 of its 45 references that are in the map. First posted on arXiv 15 Dec 2025.
Builds on
43 of 45 references.
Closest ancestors · 14
Same direction, newest first
- 2025Learning an Image Editing Model without Image Editing Pairs16 Oct
- 2025Beyond Simple Edits: X-Planner for Complex Instruction-Based Image Editing7 Jul
- 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space17 Jun
- 2024BrushEdit: All-In-One Image Inpainting and Editing13 Dec
- 2024Stable Flow: Vital Layers for Training-Free Image Editing21 Nov
- 2024Improving Long-Text Alignment for Text-to-Image Diffusion Models15 Oct
- 2024OmniGen: Unified Image Generation17 Sep
- 2023ZONE: Zero-Shot Instruction-Guided Local Editing28 Dec
- 2023SmartEdit: Exploring Complex Instruction-based Image Editing with Multimodal Large Language Models11 Dec
- 2023Improving Tuning-Free Real Image Editing with Proximal Guidance8 Jun
- 2023InstructEdit: Improving Automatic Masks for Diffusion-based Image Editing With User Instructions29 May
- 2023Localizing Object-level Shape Variations with Text-to-Image Diffusion Models20 Mar
2 more
Other references · 29
- 2023GPT-4 Technical Reportcited by 18,396
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
- 2025Qwen-Image Technical Reportcited by 1,044
- 2023Training Diffusion Models with Reinforcement Learningcited by 982
- 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 790
- 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 727
- 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 661
- 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 624
- 2023Directly Fine-Tuning Diffusion Models on Differentiable Rewardscited by 448
17 more
- 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 429
- 2023Aligning Text-to-Image Models using Human Feedbackcited by 422
- 2025DanceGRPO: Unleashing GRPO on Visual Generationcited by 413
- 2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 253
- 2024AnyEdit: Mastering Unified High-Quality Image Editing for Any Ideacited by 241
- 2023Guiding Instruction-based Image Editing via Multimodal Large Language Modelscited by 185
- 2023HIVE: Harnessing Human Feedback for Instructional Visual Editingcited by 173
- 2024GenArtist: Multimodal LLM as an Agent for Unified Image Generation and Editingcited by 133
- 2024SEED-Data-Edit Technical Report: A Hybrid Dataset for Instructional Image Editingcited by 97
- 2025Pref-GRPO: Pairwise Preference Reward-based GRPO for Stable Text-to-Image Reinforcement Learningcited by 92
- 2025KRIS-Bench: Benchmarking Next-Level Intelligent Image Editing Modelscited by 71
- 2024Deep Reward Supervisions for Tuning Text-to-Image Diffusion Modelscited by 59
- 2025KV-Edit: Training-Free Image Editing for Precise Background Preservationcited by 49
- 2025Optimizing Safe and Aligned Language Generation: A Multi-Objective GRPO Approachcited by 24
- 2023Hierarchical Diffusion Autoencoders and Disentangled Image Manipulationcited by 18
- 2025Perception Before Reasoning: Two-Stage Reinforcement Learning for Visual Reasoning in Vision-Language Modelscited by 17
- 2025Why Tree-Style Branching Matters for Thought Advantage Estimation in GRPOcited by 10
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025Uniworld-V2: Reinforce Image Editing with Diffusion Negative-aware Finetuning and MLLM Implicit Feedback
- 2025Disentangling Instruction Influence in Diffusion Transformers for Parallel Multi-Instruction-Guided Image Editing
- 2026Region-Constrained Group Relative Policy Optimization for Flow-Based Image Editing
- 2026MCIE: Multimodal LLM-Driven Complex Instruction Image Editing with Spatial Guidance
- 2026Semantically Aligned Gradient-Driven Context-Preserving Image Editing
- 2026Scalable Minimal-Change Learning for Controllable Image Editing
- 2025Training-Free Reward-Guided Image Editing via Trajectory Optimal Control
- 2025Lego-Edit: A General Image Editing Framework with Model-Level Bricks and MLLM Builder