mlsanityData to 14 Oct 2026

Multi-Axis Max@K Reinforcement Learning for Representative Diversity in Text-to-Image Generation

arXiv 2607.14962 ↗First posted 16 Jul 2026cs.LG cs.AI cs.CVcited by 2

On the map: Deep reinforcement learning methods → RL with diffusion and flow models

Abstract (arXiv)

Text-to-image (T2I) models can synthesize realistic, prompt-aligned images, yet samples generated for the same prompt often cover only a small subset of visually distinct modes. This limits diversity and, for person-centric prompts, can reflect or amplify demographic skew. We formalize this problem as target-mode coverage, the coverage of a predefined set of semantically specified modes, and propose multi-axis max@K, a group-based reinforcement learning objective for improving it in diffusion-based T2I models. Given a group of samples and one score per target mode, multi-axis max@K first takes the maximum score across samples for each mode and then sums these per-mode maxima. The resulting credit assignment gives a sample positive weight on a mode only when it raises that mode's group maximum, so different samples can contribute to different modes. We validate the credit-assignment mechanism on a synthetic mixture and on SD3.5-M with deterministic pixel-based color rewards, and then apply the same objective to perceived-appearance fairness. On held-out prompts, multi-axis max@K improves the Fairness Score by 0.23-0.36 over the base model under three automatic evaluators, while maintaining image quality and text alignment. Code is available at https://github.com/KuOnoda/multi-axis-maxk.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in RL with diffusion and flow models, one of 15 directions in Deep reinforcement learning methods. Builds on 61 of its 66 references that are in the map. First posted on arXiv 16 Jul 2026.

Builds on

61 of 66 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026Beyond the Dirac Delta: Mitigating Diversity Collapse in Reinforcement Fine-Tuning for Versatile Image Generation18 Jan

Other references · 60

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2025Qwen2.5-VL Technical Reportcited by 5,587
  4. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  5. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  6. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
  7. 2023Training Diffusion Models with Reinforcement Learningcited by 982
  8. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 977
  9. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 944
  10. 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 790
  11. 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 661
  12. 2023DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Modelscited by 450
38 more
  1. 2020Reliable Fidelity and Diversity Metrics for Generative Modelscited by 434
  2. 2025DanceGRPO: Unleashing GRPO on Visual Generationcited by 413
  3. 2018Assessing Generative Models via Precision and Recallcited by 406
  4. 2023Unified Concept Editing in Diffusion Modelscited by 388
  5. 2025Improving Video Generation with Human Feedbackcited by 284
  6. 2022Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scalecited by 262
  7. 2023Stable Bias: Analyzing Societal Representations in Diffusion Modelscited by 226
  8. 2023HIVE: Harnessing Human Feedback for Instructional Visual Editingcited by 173
  9. 2023Holistic Evaluation of Text-To-Image Modelscited by 160
  10. 2023Fair Diffusion: Instructing Text-to-Image Generation Models on Fairnesscited by 151
  11. 2023Debiasing Vision-Language Models via Biased Promptscited by 149
  12. 2025Pass@k Training for Adaptively Balancing Exploration and Exploitation of Large Reasoning Modelscited by 138
  13. 2023CADS: Unleashing the Diversity of Diffusion Models through Condition-Annealed Samplingcited by 129
  14. 2025TempFlow-GRPO: When Timing Matters for GRPO in Flow Modelscited by 97
  15. 2023Finetuning Text-to-Image Diffusion Models for Fairnesscited by 92
  16. 2024Inference-Aware Fine-Tuning for Best-of-N Sampling in Large Language Modelscited by 89
  17. 2025Pass@K Policy Optimization: Solving Harder Reinforcement Learning Problemscited by 87
  18. 2023Self-Discovering Interpretable Diffusion Latent Directions for Responsible Text-to-Image Generationcited by 69
  19. 2025RewardDance: Reward Scaling in Visual Generationcited by 68
  20. 2023ITI-GEN: Inclusive Text-to-Image Generationcited by 64
  21. 2024OpenBias: Open-set Bias Detection in Text-to-Image Generative Modelscited by 61
  22. 2025Optimizing Language Models for Inference Time Objectives using Reinforcement Learningcited by 53
  23. 2023TIBET: Identifying and Evaluating Biases in Text-to-Image Generative Modelscited by 52
  24. 2024Balancing Act: Distribution-Guided Debiasing in Diffusion Modelscited by 52
  25. 2024Improving Dynamic Object Interactions in Text-to-Video Generation with AI Feedbackcited by 47
  26. 2025Inference-Time Text-to-Video Alignment with Diffusion Latent Beam Searchcited by 41
  27. 2024Consistency-diversity-realism Pareto fronts of conditional image generative modelscited by 36
  28. 2023Fair Text-to-Image Diffusion via Fair Mappingcited by 31
  29. 2024Rethinking Training for De-biasing Text-to-Image Generation: Unlocking the Potential of Stable Diffusioncited by 22
  30. 2025DiverseGRPO: Mitigating Mode Collapse in Image Generation via Diversity-Aware GRPOcited by 17
  31. 2025The Best of N Worlds: Aligning Reinforcement Learning with Best-of-N Sampling via max@k Optimisationcited by 16
  32. 2026SetPO: Set-Level Policy Optimization for Diversity-Preserving LLM Reasoningcited by 11
  33. 2025MultiBanana: A Challenging Benchmark for Multi-Reference Text-to-Image Generationcited by 9
  34. 2024FairGen: Enhancing Fairness in Text-to-Image Diffusion Models via Self-Discovering Latent Directionscited by 8
  35. 2026From Scale to Speed: Adaptive Test-Time Scaling for Image Editingcited by 8
  36. 2026Vector Policy Optimization: Training for Diversity Improves Test-Time Searchcited by 8
  37. 2025Polychromic Objectives for Reinforcement Learningcited by 7
  38. 2025FairImagen: Post-Processing for Bias Mitigation in Text-to-Image Modelscited by 7

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026AutoRef: Harness Optimization for Agentic Multi-Reference Image Generationcited by 0
  2. 2026VIF-Bench: Evaluating Visual Instruction Following in Multi-Reference Image Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Resolving the Identity Crisis in Text-to-Image Generation
  2. 2025Learning to Sample Effective and Diverse Prompts for Text-to-Image Generation
  3. 2026Traversing the Satisfaction-Diversity Frontier in Text-to-Image Diffusion
  4. 2024Minority-Focused Text-to-Image Generation via Prompt Optimization
  5. 2024Preference Adaptive and Sequential Text-to-Image Generation
  6. 2024Large-scale Reinforcement Learning for Diffusion Models
  7. 2024Elucidating Optimal Reward-Diversity Tradeoffs in Text-to-Image Diffusion Models
  8. 2025Free Lunch Alignment of Text-to-Image Diffusion Models without Preference Image Pairs