mlsanityData to 7 Oct 2026

Bridging the Intention-Expression Gap: Aligning Multi-Dimensional Preferences via Hierarchical Relevance Feedback in Text-to-Image Diffusion

arXiv 2603.14936 ↗First posted 16 Mar 2026cs.CVcited by 0

On the map: Diffusion models → Diffusion and flow policies for robotics

Abstract (arXiv)

Users often possess a clear visual intent but struggle to articulate it precisely in language. This intention-expression gap makes aligning generated images with latent visual preferences a fundamental challenge in text-to-image diffusion models. Existing methods either require model training, sacrificing flexibility, or rely on textual feedback, imposing a heavy cognitive burden. Although recent training-free methods use click-based binary preference feedback to reduce user effort, they force Foundation Models (FMs) to infer preferences at the semantic level. When faced with multi-dimensional preferences, FMs suffer from inference overload and fail to identify exact preferred feature values under conflicting user signals. Consequently, a flexible framework for multi-dimensional feature alignment remains absent. To address this, we propose a Hierarchical Relevance Feedback-Driven (HRFD) framework. Recognizing that multiple features struggle to converge simultaneously, HRFD organizes them into a three-tier hierarchy and adapts relevance feedback to enforce coarse-to-fine convergence, minimizing cognitive load. To bypass FM inference overload, HRFD decouples the process into independent single-feature preference inference tasks. Furthermore, to overcome FMs' failure in identifying preferred values, HRFD employs statistical inference to quantify the distribution divergence of features between "liked" and "disliked" image sets, achieving robust and transparent preference measurement. Crucially, HRFD operates entirely within the external text space, remaining strictly training-free and model-agnostic. Extensive experiments demonstrate that HRFD effectively captures the user's true visual intent, significantly outperforming baseline approaches.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 29 of its 58 references that are in the map. First posted on arXiv 16 Mar 2026.

Builds on

29 of 58 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Training-free Diffusion Model Alignment with Sampling Demons8 Oct
  2. 2024Inference-Time Alignment of Diffusion Models with Direct Noise Optimization29 May
  3. 2023Diffusion Model Alignment Using Direct Preference Optimization21 Nov
  4. 2023Training Diffusion Models with Reinforcement Learning22 May

Other references · 25

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  5. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  6. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  7. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
  8. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 696
  9. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 496
  10. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesiscited by 386
  11. 2022Optimizing Prompts for Text-to-Image Generationcited by 204
  12. 2016Photo Aesthetics Ranking Network with Attributes and Content Adaptationcited by 122
13 more
  1. 2019Reinforcement Learning to Optimize Long-term User Engagement in Recommender Systemscited by 90
  2. 2025Test-time Alignment of Diffusion Models without Reward Over-optimizationcited by 89
  3. 2024PromptCharm: Text-to-Image Generation through Multi-modal Prompting and Refinementcited by 61
  4. 2023DreamSync: Aligning Text-to-Image Generation with Image Understanding Feedbackcited by 57
  5. 2023ConstitutionMaker: Interactively Critiquing Large Language Models by Converting Feedback into Principlescited by 33
  6. 2024IterComp: Iterative Composition-Aware Feedback Learning from Model Gallery for Text-to-Image Generationcited by 32
  7. 2023Generative Relevance Feedback with Large Language Modelscited by 19
  8. 2024PrompTHis: Visualizing the Process and Influence of Prompt Editing during Text-to-Image Creationcited by 11
  9. 2024AGFSync: Leveraging AI-Generated Feedback for Preference Optimization in Text-to-Image Generationcited by 9
  10. 2024Preference Adaptive and Sequential Text-to-Image Generationcited by 8
  11. 2026RAISE: Requirement-Adaptive Evolutionary Refinement for Training-Free Text-to-Image Alignmentcited by 6
  12. 2024RAVEL: Rare Concept Generation and Editing via Graph-driven Relational Guidancecited by 3
  13. 2026Preference-Guided Prompt Optimization for Text-to-Image Generationcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Instant Preference Alignment for Text-to-Image Diffusion Models
  2. 2026Rethinking Preference Alignment for Diffusion Models with Classifier-Free Guidance
  3. 2024Aligning Diffusion Models by Optimizing Human Utility
  4. 2024Aligning Diffusion Models with Noise-Conditioned Perception
  5. 2024Margin-aware Preference Optimization for Aligning Diffusion Models without Reference
  6. 2025PrefGen: Multimodal Preference Learning for Preference-Conditioned Image Generation
  7. 2025BalancedDPO: Adaptive Multi-Metric Alignment
  8. 2024Bridging the Gap: Aligning Text-to-Image Diffusion Models with Specific Feedback