Adaptive Prompt Elicitation for Text-to-Image Generation
Not on the map. Why?
Abstract (arXiv)
Aligning text-to-image generation with user intent remains challenging, as users frequently provide ambiguous inputs and struggle with model idiosyncrasies. We propose Adaptive Prompt Elicitation (APE), a technique that adaptively poses visual queries to help users refine prompts without extensive writing. Our technical contribution is a formulation of interactive intent inference under an information-theoretic framework. APE represents latent user intent as interpretable feature requirements using language model priors, adaptively generates visual queries, and compiles elicited requirements into effective prompts. Evaluation on IDEA-Bench and DesignBench shows that APE achieves stronger alignment with improved efficiency. A user study with 128 participants on user-defined tasks demonstrates 19.8% higher perceived alignment without increased workload. Our work contributes a principled approach to prompting that offers an effective and efficient complement to the prevailing prompt-based interaction paradigm with text-to-image models.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 51 of its 91 references that are in the map. First posted on arXiv 4 Feb 2026.
Builds on
51 of 91 references.
References · 51
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
- 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
- 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
- 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
- 2023Training Diffusion Models with Reinforcement Learningcited by 939
- 2017Deep Bayesian Active Learning with Image Datacited by 812
- 2023Simple and Controllable Music Generationcited by 480
- 2024Evaluating Text-to-Visual Generation with Image-to-Text Generationcited by 465
38 more
- 2019BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learningcited by 428
- 2022DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Modelscited by 376
- 2024Multilingual E5 Text Embeddings: A Technical Reportcited by 357
- 2023DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Datacited by 322
- 2022Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scalecited by 262
- 2021Design Guidelines for Prompt Engineering Text-to-Image Generative Modelscited by 216
- 2022Optimizing Prompts for Text-to-Image Generationcited by 204
- 2023Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMscited by 191
- 2022Explanations Can Reduce Overreliance on AI Systems During Decision-Makingcited by 111
- 2024Improving Text-to-Image Consistency via Automatic Prompt Optimizationcited by 88
- 2017Preferential Bayesian Optimizationcited by 72
- 2023Bridging the Gulf of Envisioning: Cognitive Design Challenges in LLM Interfacescited by 71
- 2023Promptify: Text-to-Image Generation through Interactive Prompt Exploration with Large Language Modelscited by 67
- 2024PromptCharm: Text-to-Image Generation through Multi-modal Prompting and Refinementcited by 61
- 2024OpenBias: Open-set Bias Detection in Text-to-Image Generative Modelscited by 61
- 2022Designing Creative AI Partners with COFI: A Framework for Modeling Interaction in Human-AI Co-Creative Systemscited by 59
- 2023PromptMagician: Interactive Prompt Engineering for Text-to-Image Creationcited by 58
- 2023PromptPaint: Steering Text-to-Image Generation Through Paint Medium-like Interactionscited by 34
- 2024Large Language Models are Biased Because They Are Large Language Modelscited by 26
- 2020Sequential Gallery for Interactive Visual Design Optimizationcited by 24
- 2023A Prompt Log Analysis of Text-to-Image Generation Systemscited by 24
- 2023Prompt Expansion for Adaptive Text-to-Image Generationcited by 24
- 2024Proactive Agents for Multi-Turn Text-to-Image Generation Under Uncertaintycited by 23
- 2025Active Task Disambiguation with LLMscited by 23
- 2024Amortized Bayesian Experimental Design for Decision-Makingcited by 20
- 2024Position: Towards Bidirectional Human-AI Alignmentcited by 15
- 2024User-Driven Value Alignment: Understanding Users' Perceptions and Strategies for Addressing Biased and Discriminatory Statements in AI Companionscited by 14
- 2024PrompTHis: Visualizing the Process and Influence of Prompt Editing during Text-to-Image Creationcited by 11
- 2025Bayesian Optimization for Controlled Image Editing via LLMscited by 11
- 2025Twin Co-Adaptive Dialogue for Progressive Image Generationcited by 11
- 2024IntentTuner: An Interactive Framework for Integrating Human Intents in Fine-tuning Text-to-Image Generative Modelscited by 8
- 2025Brickify: Enabling Expressive Design Intent Specification through Direct Manipulation on Design Tokenscited by 8
- 2025PromptMap: An Alternative Interaction Style for AI-Based Image Generationcited by 8
- 2024A User-Friendly Framework for Generating Model-Preferred Prompts in Text-to-Image Synthesiscited by 7
- 2025FontCraft: Multimodal Font Design Using Interactive Bayesian Optimizationcited by 7
- 2024MAPLE: A Framework for Active Preference Learning Guided by Large Language Modelscited by 6
- 2025SketchFlex: Facilitating Spatial-Semantic Coherence in Text-to-Image Generation with Region-Based Sketchescited by 6
- 2025Adultification Bias in LLMs and Text-to-Image Modelscited by 5
50 most cited of 51 All 51 (JSON)
Built on by
Not cited yet.