# DyMO: Training-Free Diffusion Model Alignment with Dynamic Multi-Objective Scheduling

arXiv: https://arxiv.org/abs/2412.00759 · HTML: https://mlsanity.com/paper/2412.00759/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models.
- Builds on 47 of its 55 references that are in the map.
- First posted on arXiv 1 Dec 2024.

## Placed in

- Diffusion models / [Diffusion and flow policies for robotics](https://mlsanity.com/diffusion/diffusion-and-flow-policies-for-robotics/)
- LLM alignment / RLHF / preference optimisation / [Diffusion and multimodal model alignment](https://mlsanity.com/alignment/diffusion-and-multimodal-model-alignment/)

## Builds on (most cited first, 30 of 47)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [Scaling Autoregressive Models for Content-Rich Text-to-Image Generation](https://mlsanity.com/paper/2206.10789/) (2022)
- [Training Diffusion Models with Reinforcement Learning](https://mlsanity.com/paper/2305.13301/) (2023)
- [Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation](https://mlsanity.com/paper/2305.01569/) (2023)
- [GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment](https://mlsanity.com/paper/2310.11513/) (2023)
- [Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation](https://mlsanity.com/paper/2211.12572/) (2022)
- [Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesis](https://mlsanity.com/paper/2306.09341/) (2023)
- [Diffusion Model Alignment Using Direct Preference Optimization](https://mlsanity.com/paper/2311.12908/) (2023)
- [RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment](https://mlsanity.com/paper/2304.06767/) (2023)
- [Directly Fine-Tuning Diffusion Models on Differentiable Rewards](https://mlsanity.com/paper/2309.17400/) (2023)
- [Universal Guidance for Diffusion Models](https://mlsanity.com/paper/2302.07121/) (2023)
- [Aligning Text-to-Image Models using Human Feedback](https://mlsanity.com/paper/2302.12192/) (2023)
- [Differentiable Augmentation for Data-Efficient GAN Training](https://mlsanity.com/paper/2006.10738/) (2020)
- [FreeDoM: Training-Free Energy-Guided Conditional Diffusion Model](https://mlsanity.com/paper/2303.09833/) (2023)
- [Aligning Text-to-Image Diffusion Models with Reward Backpropagation](https://mlsanity.com/paper/2310.03739/) (2023)
- [Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation](https://mlsanity.com/paper/2403.12015/) (2024)
- [Using Human Feedback to Fine-tune Diffusion Models without Any Reward Model](https://mlsanity.com/paper/2311.13231/) (2023)

## Built on by (most cited first, 27 of 27)

- [Pref-GRPO: Pairwise Preference Reward-based GRPO for Stable Text-to-Image Reinforcement Learning](https://mlsanity.com/paper/2508.20751/) (2025)
- [MIRA: Towards Mitigating Reward Hacking in Inference-Time Alignment of T2I Diffusion Models](https://mlsanity.com/paper/2510.01549/) (2025)
- [ConsistentRFT: Reducing Visual Hallucinations in Flow-based Reinforcement Fine-Tuning](https://mlsanity.com/paper/2602.03425/) (2026)
- [Do Less, Achieve More: Do We Need Every-Step Optimization for RL Fine-tuning of Diffusion Models?](https://mlsanity.com/paper/2605.15855/) (2026)
- [HyperAlign: Hypernetwork for Efficient Test-Time Alignment of Diffusion Models](https://mlsanity.com/paper/2601.15968/) (2026)
- [EasyTune: Efficient Step-Aware Fine-Tuning for Diffusion-Based Motion Generation](https://mlsanity.com/paper/2602.07967/) (2026)
- [Let Your Video Listen to Your Music!](https://mlsanity.com/paper/2506.18881/) (2025)
- [Subjective Camera 1.0: Bridging Human Cognition and Visual Reconstruction through Sequence-Aware Sketch-Guided Diffusion](https://mlsanity.com/paper/2506.23711/) (2025)
- [RewardFlow: Generate Images by Optimizing What You Reward](https://mlsanity.com/paper/2604.08536/) (2026)
- [Hierarchical and Step-Layer-Wise Tuning of Attention Specialty for Multi-Instance Synthesis in Diffusion Transformers](https://mlsanity.com/paper/2504.10148/) (2025)
- [When Distillation Breaks Motion Control: Restoring Generative Trajectories for Fast Video Generators](https://mlsanity.com/paper/2506.19348/) (2025)
- [AGSwap: Overcoming Category Boundaries in Object Fusion via Adaptive Group Swapping](https://mlsanity.com/paper/2509.18699/) (2025)
- [NEGATE: Constrained Semantic Guidance for Linguistic Negation in Text-to-Video Diffusion](https://mlsanity.com/paper/2603.06533/) (2026)
- [Rethinking Test Time Scaling for Flow-Matching Generative Models](https://mlsanity.com/paper/2511.22242/) (2025)
- [Early Failure Detection and Intervention in Video Diffusion Models](https://mlsanity.com/paper/2603.14320/) (2026)
- [MotionRFT: Unified Reinforcement Fine-Tuning for Text-to-Motion Generation](https://mlsanity.com/paper/2603.27185/) (2026)
- [DCR: Counterfactual Attractor Guidance for Rare Compositional Generation](https://mlsanity.com/paper/2605.06512/) (2026)
- [Explore or Converge? Stage-Guided Per-Step Optimization for Diffusion Models](https://mlsanity.com/paper/2608.06768/) (2026)
- [Test-Time Weak-to-Strong Alignment: Transferring Implicit Rewards from Weak to Strong Flow Models](https://mlsanity.com/paper/2609.05968/) (2026)
- [SIPO: Stabilized and Improved Preference Optimization for Aligning Diffusion Models](https://mlsanity.com/paper/2505.21893/) (2025)
- [Test-Time Alignment of Text-to-Image Diffusion Models via Null-Text Embedding Optimisation](https://mlsanity.com/paper/2511.20889/) (2025)
- [FreeInpaint: Tuning-free Prompt Alignment and Visual Rationality Enhancement in Image Inpainting](https://mlsanity.com/paper/2512.21104/) (2025)
- [CritiFusion: Semantic Critique and Spectral Alignment for Faithful Text-to-Image Generation](https://mlsanity.com/paper/2512.22681/) (2025)
- [TPD: Temporal Prior Decoupling for Text-to-Video Diffusion Models](https://mlsanity.com/paper/2607.26706/) (2026)
- [PAST: Prompt-Adaptive Sampling Termination for Efficient Diffusion Model](https://mlsanity.com/paper/2608.06794/) (2026)
- [SwiftExplorer: Training-free Diffusion Model Alignment with Swift Diversity Exploration](https://mlsanity.com/paper/2609.06651/) (2026)
- [Panoptic Scene Program Diffusion Transformer](https://mlsanity.com/paper/2609.31780/) (2026)

## Abstract (arXiv)

Text-to-image diffusion model alignment is critical for improving the alignment between the generated images and human preferences. While training-based methods are constrained by high computational costs and dataset requirements, training-free alignment methods remain underexplored and are often limited by inaccurate guidance. We propose a plug-and-play training-free alignment method, DyMO, for aligning the generated images and human preferences during inference. Apart from text-aware human preference scores, we introduce a semantic alignment objective for enhancing the semantic alignment in the early stages of diffusion, relying on the fact that the attention maps are effective reflections of the semantics in noisy images. We propose dynamic scheduling of multiple objectives and intermediate recurrent steps to reflect the requirements at different steps. Experiments with diverse pre-trained diffusion models and metrics demonstrate the effectiveness and robustness of the proposed method.