Personalized Image Editing in Text-to-Image Diffusion Models via Collaborative Direct Preference Optimization
On the map: Diffusion models → Image generation, text-to-image and editing
also in 1 more
Abstract (arXiv)
Text-to-image (T2I) diffusion models have made remarkable strides in generating and editing high-fidelity images from text. Yet, these models remain fundamentally generic, failing to adapt to the nuanced aesthetic preferences of individual users. In this work, we present the first framework for personalized image editing in diffusion models, introducing Collaborative Direct Preference Optimization (C-DPO), a novel method that aligns image edits with user-specific preferences while leveraging collaborative signals from like-minded individuals. Our approach encodes each user as a node in a dynamic preference graph and learns embeddings via a lightweight graph neural network, enabling information sharing across users with overlapping visual tastes. We enhance a diffusion model's editing capabilities by integrating these personalized embeddings into a novel DPO objective, which jointly optimizes for individual alignment and neighborhood coherence. Comprehensive experiments, including user studies and quantitative benchmarks, demonstrate that our method consistently outperforms baselines in generating edits that are aligned with user preferences.
Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 34 of its 41 references that are in the map. First posted on arXiv 6 Nov 2025.
Builds on
34 of 41 references.
Closest ancestors · 8
Same direction, newest first
- 2024Semantic Image Inversion and Editing using Rectified Stochastic Differential Equations14 Oct
- 2024TurboEdit: Text-Based Image Editing Using Few-Step Diffusion Models1 Aug
- 2023LEDITS++: Limitless Image Editing using Text-to-Image Models28 Nov
- 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructions17 Nov
- 2022Blended Latent Diffusion6 Jun
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
Other references · 26
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2023GPT-4 Technical Reportcited by 18,132
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
14 more
- 2017Inductive Representation Learning on Large Graphscited by 6,284
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
- 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
- 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 847
- 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 767
- 2023Making LLaMA SEE and Draw with SEED Tokenizercited by 184
- 2024Personalized Language Modeling from Personalized Human Feedbackcited by 124
- 2025Personalized Preference Fine-tuning of Diffusion Modelscited by 22
- 2024ViPer: Visual Personalization of Generative Models via Individual Preference Learningcited by 17
- 2024Personalized Image Generation with Large Multimodal Modelscited by 14
- 2022Layer-refined Graph Convolutional Networks for Recommendationcited by 11
- 2025Nexus-Gen: Unified Image Understanding, Generation, and Editing via Prefilled Autoregression in Shared Embedding Spacecited by 9
- 2021Pre-training Recommender Systems via Reinforced Attentive Multi-relational Graph Neural Networkcited by 7
Built on by
Citing papers per year: 2025: 1; 2026: 5.
6 papers.
- 2025Personalized Reward Modeling for Text-to-Image Generationcited by 3
- 2026Premier: Personalized Preference Modulation with Learnable User Embedding in Text-to-Image Generationcited by 2
- 2026Offline Preference Optimization for Rectified Flow with Noise-Tracked Pairscited by 1
- 2026Personalizing Text-to-Image Generation to Individual Tastecited by 0
- 2026RankT2I: A Submodular Framework for Discovering Interpretable and Diverse Semantics in Text-to-Image Modelscited by 0
- 2026Personalized Image Generation with Reasoning and Reflectioncited by 0
- 2026Personalized Image Generation with Reasoning and Reflection30 Sep 2026
- 2026RankT2I: A Submodular Framework for Discovering Interpretable and Diverse Semantics in Text-to-Image Models14 Aug 2026
- 2026Offline Preference Optimization for Rectified Flow with Noise-Tracked Pairs10 May 2026
- 2026Personalizing Text-to-Image Generation to Individual Taste8 Apr 2026
- 2026Premier: Personalized Preference Modulation with Learnable User Embedding in Text-to-Image Generation21 Mar 2026
- 2025Personalized Reward Modeling for Text-to-Image Generation21 Nov 2025
Similar, not cited
Close in wording, no citation either way
- 2023Custom-Edit: Text-Guided Image Editing with Customized Diffusion Models
- 2025Energy-Guided Optimization for Personalized Image Editing with Pretrained Text-to-Image Diffusion Models
- 2024Margin-aware Preference Optimization for Aligning Diffusion Models without Reference
- 2025Diffusion-SDPO: Safeguarded Direct Preference Optimization for Diffusion Models
- 2025Preserve and Personalize: Personalized Text-to-Image Diffusion Models without Distributional Drift
- 2025Preference Alignment on Diffusion Model: A Comprehensive Survey for Image Generation and Editing
- 2023Textual and Visual Prompt Fusion for Image Editing via Step-Wise Alignment
- 2025Towards Better Optimization For Listwise Preference in Diffusion Models