mlsanityData to 7 Oct 2026

Personalized Image Editing in Text-to-Image Diffusion Models via Collaborative Direct Preference Optimization

arXiv 2511.05616 ↗First posted 6 Nov 2025cs.CV cs.AIcited by 6

On the map: Diffusion models → Image generation, text-to-image and editing

also in 1 more
Abstract (arXiv)

Text-to-image (T2I) diffusion models have made remarkable strides in generating and editing high-fidelity images from text. Yet, these models remain fundamentally generic, failing to adapt to the nuanced aesthetic preferences of individual users. In this work, we present the first framework for personalized image editing in diffusion models, introducing Collaborative Direct Preference Optimization (C-DPO), a novel method that aligns image edits with user-specific preferences while leveraging collaborative signals from like-minded individuals. Our approach encodes each user as a node in a dynamic preference graph and learns embeddings via a lightweight graph neural network, enabling information sharing across users with overlapping visual tastes. We enhance a diffusion model's editing capabilities by integrating these personalized embeddings into a novel DPO objective, which jointly optimizes for individual alignment and neighborhood coherence. Comprehensive experiments, including user studies and quantitative benchmarks, demonstrate that our method consistently outperforms baselines in generating edits that are aligned with user preferences.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 34 of its 41 references that are in the map. First posted on arXiv 6 Nov 2025.

Builds on

34 of 41 references.

Closest ancestors · 8

Same direction, newest first

  1. 2024Semantic Image Inversion and Editing using Rectified Stochastic Differential Equations14 Oct
  2. 2024TurboEdit: Text-Based Image Editing Using Few-Step Diffusion Models1 Aug
  3. 2023LEDITS++: Limitless Image Editing using Text-to-Image Models28 Nov
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
  5. 2022InstructPix2Pix: Learning to Follow Image Editing Instructions17 Nov
  6. 2022Blended Latent Diffusion6 Jun
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr

Other references · 26

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  7. 2022Training language models to follow instructions with human feedbackcited by 15,712
  8. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  9. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  10. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  11. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  12. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
14 more
  1. 2017Inductive Representation Learning on Large Graphscited by 6,284
  2. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  3. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  4. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  5. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 847
  6. 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 767
  7. 2023Making LLaMA SEE and Draw with SEED Tokenizercited by 184
  8. 2024Personalized Language Modeling from Personalized Human Feedbackcited by 124
  9. 2025Personalized Preference Fine-tuning of Diffusion Modelscited by 22
  10. 2024ViPer: Visual Personalization of Generative Models via Individual Preference Learningcited by 17
  11. 2024Personalized Image Generation with Large Multimodal Modelscited by 14
  12. 2022Layer-refined Graph Convolutional Networks for Recommendationcited by 11
  13. 2025Nexus-Gen: Unified Image Understanding, Generation, and Editing via Prefilled Autoregression in Shared Embedding Spacecited by 9
  14. 2021Pre-training Recommender Systems via Reinforced Attentive Multi-relational Graph Neural Networkcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 5.

6 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2023Custom-Edit: Text-Guided Image Editing with Customized Diffusion Models
  2. 2025Energy-Guided Optimization for Personalized Image Editing with Pretrained Text-to-Image Diffusion Models
  3. 2024Margin-aware Preference Optimization for Aligning Diffusion Models without Reference
  4. 2025Diffusion-SDPO: Safeguarded Direct Preference Optimization for Diffusion Models
  5. 2025Preserve and Personalize: Personalized Text-to-Image Diffusion Models without Distributional Drift
  6. 2025Preference Alignment on Diffusion Model: A Comprehensive Survey for Image Generation and Editing
  7. 2023Textual and Visual Prompt Fusion for Image Editing via Step-Wise Alignment
  8. 2025Towards Better Optimization For Listwise Preference in Diffusion Models