mlsanityData to 7 Oct 2026

Diffusion and multimodal model alignment

709 papers2 this weekMedium confidence in this grouping: mostly one topic, with blurry edgesMedium confidence in this grouping: mostly one topic, with blurry edges · About

Keywords: preference, reward, diffusion, generation, image

Key facts. Diffusion and multimodal model alignment: 709 papers on arXiv since 2017 (data through 7 Oct 2026). 230 of its 709 mapped papers (32%) cite “Diffusion Model Alignment Using Direct Preference Optimization” (2023, arXiv 2311.12908). 144 of its 709 mapped papers (20%) cite “Directly Fine-Tuning Diffusion Models on Differentiable Rewards” (2023, arXiv 2309.17400). 20 new papers 17 Sep–7 Oct.

What’s new

17 Sep – 7 Oct 2026

20 new papers, 17 Sep – 7 Oct; 2 this week.

  1. This week · 2
  2. Enabling Preference-driven Unlearning in Few-step Distilled Text-to-Image Diffusion Models
  3. Personalize at Test Time: Learning User Preferences for Image Generation
  4. Earlier · 18
  5. MGPO: Manifold-Guided Diffusion Alignment for Task-Aware Dataset Distillation
  6. Best-of-N Guidance for Test-time Diffusion Alignment
  7. Geometry-Aware Preference Optimization for Text-to-Image Diffusion Models
  8. FASTER: Fast Adjoint Stochastic Transport for Endpoint Refinement in Reward-Guided Image Editing
  9. DEPICT: Scoring Text-to-Image Alignment by Answer Agreement
  10. Post-Training Frontier Text-to-Image Models by Composing Preference and Rubric Rewards
  11. Discrete Annotation, Continuous Preference: Rethinking Supervision for Accurate and Generalizable Aesthetic Image Cropping
  12. Manifold-Constrained Initial Noise Optimization for Efficient Generative Model Alignment
All 20
  1. Think Before You Score: Thinking Reward Model for Visual Generation
  2. Beyond Binary Preferences: Graded Preference Optimization for Limb-Motion Captioning
  3. Diffusion Reward Models also fits: Reward modeling and RLHF
  4. CoRe: Co-Evolving Reward Models for Mitigating Latent Reward Hacking in Video Diffusion Models
  5. Verifiable Visual Rewards Transfer from Synthetic Scenes to Natural Prompts
  6. Beyond Saying Less: Fine-Grained Alignment for Informative and Faithful Vision-Language Models
  7. FestDPO: Few-step Generator Alignment with Direct Preference Optimization
  8. Aligning One-Step Generative Models with Reward-Weighted Transport Distillation
  9. RULER: Instance-aware Rubric Rewards for SVG Generation also fits: Reward hacking and misspecification
  10. AnyviewMeter: Adapting Robotic Reward Models with Camera Geometry and Multi-View Attention also fits: Reward modeling and RLHF

Foundations

Most cited by papers here

  1. Diffusion Model Alignment Using Direct Preference Optimization2023cited by 230 here
  2. Directly Fine-Tuning Diffusion Models on Differentiable Rewards2023cited by 144 here
  3. Aligning Text-to-Image Models using Human Feedback2023cited by 114 here
  4. Aligning Large Multimodal Models with Factually Augmented RLHF2023cited by 113 here
  5. Using Human Feedback to Fine-tune Diffusion Models without Any Reward Model2023cited by 113 here

Lineage

The papers this direction builds on, oldest first.

Each paper on the line cites the one before.

  1. 2017Deep reinforcement learning from human preferencescited by 121 here
  2. 2019Fine-Tuning Language Models from Human Preferencescited by 74 here
  3. 2020Learning to summarize from human feedbackcited by 72 here
  4. 2022Training language models to follow instructions with human feedbackcited by 294 here
  5. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 86 here
  6. 2023Aligning Text-to-Image Models using Human Feedbackcited by 114 here
  7. 2023Directly Fine-Tuning Diffusion Models on Differentiable Rewardscited by 144 here
  8. 2024VisionReward: Fine-Grained Multi-Dimensional Human Preference Learning for Image and Video Generationcited by 52 here
  9. 2025Improving Video Generation with Human Feedbackcited by 77 here
  10. 2025DanceGRPO: Unleashing GRPO on Visual Generationcited by 97 here
  11. 2025HPSv3: Towards Wide-Spectrum Human Preference Scorecited by 55 here
13 more it builds on, not in this chain
  1. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 49 here
  2. 2023Human Preference Score: Better Aligning Text-to-Image Models with Human Preferencecited by 77 here
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 382 here
  4. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 113 here
  5. 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 52 here
  6. 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 230 here
  7. 2023Using Human Feedback to Fine-tune Diffusion Models without Any Reward Modelcited by 113 here
  8. 2023Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimizationcited by 56 here
  9. 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedbackcited by 91 here
  10. 2024Aligning Modalities in Vision Large Language Models via Preference Fine-tuningcited by 68 here
  11. 2024Aligning Diffusion Models by Optimizing Human Utilitycited by 58 here
  12. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 48 here
  13. 2025Unified Reward Model for Multimodal Understanding and Generationcited by 65 here

The 24 most cited of 835.

Browse

30 of 709, ranked 6–35

  1. 6DanceGRPO: Unleashing GRPO on Visual Generation202597
  2. 7RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback202391
  3. 8Improving Video Generation with Human Feedback202577
  4. 9Human Preference Score: Better Aligning Text-to-Image Models with Human Preference202377
  5. 10Aligning Modalities in Vision Large Language Models via Preference Fine-tuning202468
  6. 11Unified Reward Model for Multimodal Understanding and Generation202565
  7. 12Aligning Diffusion Models by Optimizing Human Utility202458
  8. 13Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimization202356
  9. 14HPSv3: Towards Wide-Spectrum Human Preference Score202555
  10. 15VisionReward: Fine-Grained Multi-Dimensional Human Preference Learning for Image and Video Generation202452
All 30
  1. 16VideoDPO: Omni-Preference Alignment for Video Diffusion Generation202447
  2. 17MixGRPO: Unlocking Flow-based GRPO Efficiency with Mixed ODE-SDE202543
  3. 18Silkie: Preference Distillation for Large Visual Language Models202341
  4. 19Video Diffusion Alignment via Reward Gradients202440
  5. 20Aesthetic Post-Training Diffusion Models from Generic Preferences with Step-by-step Preference Optimization202439
  6. 21mDPO: Conditional Preference Optimization for Multimodal Large Language Models202434
  7. 22Unified Multimodal Chain-of-Thought Reward Model through Reinforcement Fine-Tuning202533
  8. 23Rich Human Feedback for Text-to-Image Generation202333
  9. 24Margin-aware Preference Optimization for Aligning Diffusion Models without Reference202430
  10. 25Self-Play Fine-Tuning of Diffusion Models for Text-to-Image Generation202429
  11. 26RLAIF-V: Open-Source AI Feedback Leads to Super GPT-4V Trustworthiness202429
  12. 27Strengthening Multimodal Large Language Model with Bootstrapped Preference Optimization202428
  13. 28VL-RewardBench: A Challenging Benchmark for Vision-Language Generative Reward Models202428
  14. 29A Dense Reward View on Aligning Text-to-Image Diffusion with Preference202427
  15. 30InstructVideo: Instructing Video Diffusion Models with Human Feedback202327
  16. 31InternLM-XComposer2.5-Reward: A Simple Yet Effective Multi-Modal Reward Model202526
  17. 32CLIP-DPO: Vision-Language Models as a Source of Preference for Fixing Hallucinations in LVLMs202426
  18. 33LiFT: Leveraging Human Feedback for Text-to-Video Model Alignment202426
  19. 34TempFlow-GRPO: When Timing Matters for GRPO in Flow Models202525
  20. 35Deep Reward Supervisions for Tuning Text-to-Image Diffusion Models202425