SaTPO: Text-to-Image Safety Alignment via Transition Preference Optimization
On the map: Diffusion models → Image generation, text-to-image and editing also fits: Robustness, privacy and adversarial aspects
also in 1 more
Abstract (arXiv)
Safety alignment of text-to-image diffusion models aims to suppress harmful content without compromising useful generation capabilities. However, preferences over completed images compress multidimensional generation factors into a coarse signal, potentially driving changes beyond those required for safety. To address this, we propose Safety Transition Preference Optimization (SaTPO), which enforces step-wise alignment during denoising. SaTPO leverages the insight that an unsafe prompt and its safe rewrite naturally form a tightly coupled preference pair when conditioned on a shared parent state and noise realization. By anchoring optimization on identical parent states, SaTPO eliminates historical trajectory divergence, confining safety adaptation to an incremental alignment while preserving task-relevant semantics. Multi-scale drift constraints and benign replay further limit behavioral deviation and preserve ordinary generation. We additionally construct CoProPV, a refined dataset ensuring visually verified contrasts. Experiments on Stable Diffusion v1.5 and SDXL demonstrate improved safety-quality trade-offs. Compared with AlignGuard, PerspectiveVision unsafe-detection rates on Inappropriate Image Prompts decrease from 12.70% to 6.00% and from 15.04% to 9.30%, respectively, while FID improves from 32.28 to 25.95 and from 36.50 to 27.51. Warning: this paper includes potentially offensive content.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 36 of its 44 references that are in the map. First posted on arXiv 14 Oct 2026.
Builds on
36 of 44 references.
Closest ancestors · 1
Same direction, newest first
Other references · 35
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2020Denoising Diffusion Implicit Modelscited by 9,781
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2022Classifier-Free Diffusion Guidancecited by 5,847
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
- 2023Training Diffusion Models with Reinforcement Learningcited by 982
- 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 790
- 2023Erasing Concepts from Diffusion Modelscited by 539
- 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 513
23 more
- 2023DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Modelscited by 450
- 2023Unified Concept Editing in Diffusion Modelscited by 388
- 2024MACE: Mass Concept Erasure in Diffusion Modelscited by 307
- 2023Using Human Feedback to Fine-tune Diffusion Models without Any Reward Modelcited by 252
- 2023Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models?cited by 250
- 2023MMA-Diffusion: MultiModal Attack on Diffusion Modelscited by 194
- 2023SneakyPrompt: Jailbreaking Text-to-image Generative Modelscited by 189
- 2023Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Modelscited by 175
- 2022Can Machines Help Us Answering Question 16 in Datasheets, and In Turn Reflecting on Inappropriate Content?cited by 155
- 2024Reliable and Efficient Concept Erasure of Text-to-Image Diffusion Modelscited by 133
- 2024SAFREE: Training-Free and Adaptive Guard for Safe Text-to-Image And Video Generationcited by 133
- 2023Circumventing Concept Erasure Methods For Text-to-Image Generative Modelscited by 85
- 2024Aesthetic Post-Training Diffusion Models from Generic Preferences with Step-by-step Preference Optimizationcited by 84
- 2024Latent Guard: a Safety Framework for Text-to-image Generationcited by 71
- 2024SafeGen: Mitigating Sexually Explicit Content Generation in Text-to-Image Modelscited by 66
- 2025Diffusion Model as a Noise-Aware Latent Reward Model for Step-Level Preference Optimizationcited by 37
- 2024Adversarial Attacks and Defenses on Text-to-Image Diffusion Models: A Surveycited by 35
- 2024Universal Prompt Optimizer for Safe Text-to-Image Generationcited by 28
- 2025Diffusion-NPO: Negative Preference Optimization for Better Preference Aligned Generation of Diffusion Modelscited by 23
- 2024AlignGuard: Scalable Safety Alignment for Text-to-Image Generationcited by 20
- 2024Explaining generative diffusion models via visual analysis for interpretable decision-making processcited by 17
- 2025Diffusion-SDPO: Safeguarded Direct Preference Optimization for Diffusion Modelscited by 7
- 2025Direct Distributional Optimization for Provable Alignment of Diffusion Modelscited by 5
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025SPQR: A Multi-Dimensional Benchmark for Safety Alignment under Benign Model Adaptation
- 2025The Path to Reconciling Quality and Safety in Text-to-Image Generation: Dataset, Method, and Evaluation
- 2025Alignment and Safety of Diffusion Models via Reinforcement Learning and Reward Modeling: A Survey
- 2025Personalized Safety Alignment for Text-to-Image Diffusion Models
- 2025BalancedDPO: Adaptive Multi-Metric Alignment
- 2024Safety Alignment Backfires: Preventing the Re-emergence of Suppressed Concepts in Fine-tuned Text-to-Image Diffusion Models
- 2026Alignment-Guided Score Matching for Text-to-Image Alignment in Diffusion Models
- 2026SPOT: Selective Prompt Projection via Total Variation for Inference-Only Safe Text-to-Image Generation