# Diffusion and multimodal model alignment

Area: [LLM alignment / RLHF / preference optimisation](https://mlsanity.com/alignment/index.md). HTML: https://mlsanity.com/alignment/diffusion-and-multimodal-model-alignment/

## Key facts

- Diffusion and multimodal model alignment: 709 papers on arXiv since 2017 (data through 7 Oct 2026).
- 230 of its 709 mapped papers (32%) cite “Diffusion Model Alignment Using Direct Preference Optimization” (2023, arXiv 2311.12908).
- 144 of its 709 mapped papers (20%) cite “Directly Fine-Tuning Diffusion Models on Differentiable Rewards” (2023, arXiv 2309.17400).
- 20 new papers 17 Sep–7 Oct.

## Most cited here

- [Diffusion Model Alignment Using Direct Preference Optimization](https://mlsanity.com/paper/2311.12908/) (2023), cited by 230 papers here
- [Directly Fine-Tuning Diffusion Models on Differentiable Rewards](https://mlsanity.com/paper/2309.17400/) (2023), cited by 144 papers here
- [Aligning Text-to-Image Models using Human Feedback](https://mlsanity.com/paper/2302.12192/) (2023), cited by 114 papers here
- [Aligning Large Multimodal Models with Factually Augmented RLHF](https://mlsanity.com/paper/2309.14525/) (2023), cited by 113 papers here
- [Using Human Feedback to Fine-tune Diffusion Models without Any Reward Model](https://mlsanity.com/paper/2311.13231/) (2023), cited by 113 papers here
- [DanceGRPO: Unleashing GRPO on Visual Generation](https://mlsanity.com/paper/2505.07818/) (2025), cited by 97 papers here
- [RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback](https://mlsanity.com/paper/2312.00849/) (2023), cited by 91 papers here
- [Human Preference Score: Better Aligning Text-to-Image Models with Human Preference](https://mlsanity.com/paper/2303.14420/) (2023), cited by 77 papers here
- [Improving Video Generation with Human Feedback](https://mlsanity.com/paper/2501.13918/) (2025), cited by 77 papers here
- [Aligning Modalities in Vision Large Language Models via Preference Fine-tuning](https://mlsanity.com/paper/2402.11411/) (2024), cited by 68 papers here

## New (17 Sep–7 Oct)

- [Enabling Preference-driven Unlearning in Few-step Distilled Text-to-Image Diffusion Models](https://mlsanity.com/paper/2610.10859/) (2026-10-07)
- [Personalize at Test Time: Learning User Preferences for Image Generation](https://mlsanity.com/paper/2610.09015/) (2026-10-06)
- [MGPO: Manifold-Guided Diffusion Alignment for Task-Aware Dataset Distillation](https://mlsanity.com/paper/2610.05252/) (2026-10-04)
- [Best-of-N Guidance for Test-time Diffusion Alignment](https://mlsanity.com/paper/2610.05108/) (2026-10-04)
- [Geometry-Aware Preference Optimization for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2610.04980/) (2026-10-04)
- [FASTER: Fast Adjoint Stochastic Transport for Endpoint Refinement in Reward-Guided Image Editing](https://mlsanity.com/paper/2610.04538/) (2026-10-03)
- [DEPICT: Scoring Text-to-Image Alignment by Answer Agreement](https://mlsanity.com/paper/2610.03617/) (2026-10-02)
- [Post-Training Frontier Text-to-Image Models by Composing Preference and Rubric Rewards](https://mlsanity.com/paper/2610.02967/) (2026-10-02)
- [Discrete Annotation, Continuous Preference: Rethinking Supervision for Accurate and Generalizable Aesthetic Image Cropping](https://mlsanity.com/paper/2610.00582/) (2026-09-30)
- [Manifold-Constrained Initial Noise Optimization for Efficient Generative Model Alignment](https://mlsanity.com/paper/2610.00365/) (2026-09-30)
- [Think Before You Score: Thinking Reward Model for Visual Generation](https://mlsanity.com/paper/2609.37372/) (2026-09-29)
- [Beyond Binary Preferences: Graded Preference Optimization for Limb-Motion Captioning](https://mlsanity.com/paper/2609.36628/) (2026-09-29)
- [Diffusion Reward Models](https://mlsanity.com/paper/2609.33803/) (2026-09-29)
- [CoRe: Co-Evolving Reward Models for Mitigating Latent Reward Hacking in Video Diffusion Models](https://mlsanity.com/paper/2609.36245/) (2026-09-28)
- [Verifiable Visual Rewards Transfer from Synthetic Scenes to Natural Prompts](https://mlsanity.com/paper/2609.35641/) (2026-09-28)
- [Beyond Saying Less: Fine-Grained Alignment for Informative and Faithful Vision-Language Models](https://mlsanity.com/paper/2609.35294/) (2026-09-28)
- [FestDPO: Few-step Generator Alignment with Direct Preference Optimization](https://mlsanity.com/paper/2609.34673/) (2026-09-28)
- [Aligning One-Step Generative Models with Reward-Weighted Transport Distillation](https://mlsanity.com/paper/2609.30840/) (2026-09-25)
- [RULER: Instance-aware Rubric Rewards for SVG Generation](https://mlsanity.com/paper/2609.25270/) (2026-09-21)
- [AnyviewMeter: Adapting Robotic Reward Models with Camera Geometry and Multi-View Attention](https://mlsanity.com/paper/2609.20106/) (2026-09-17)