mlsanityData to 7 Oct 2026

Tailored Visions: Enhancing Text-to-Image Generation with Personalized Prompt Rewriting

arXiv 2310.08129 ↗First posted 12 Oct 2023cs.CVcited by 23

Not on the map. Why?

Abstract (arXiv)

Despite significant progress in the field, it is still challenging to create personalized visual representations that align closely with the desires and preferences of individual users. This process requires users to articulate their ideas in words that are both comprehensible to the models and accurately capture their vision, posing difficulties for many users. In this paper, we tackle this challenge by leveraging historical user interactions with the system to enhance user prompts. We propose a novel approach that involves rewriting user prompts based on a newly collected large-scale text-to-image dataset with over 300k prompts from 3115 users. Our rewriting model enhances the expressiveness and alignment of user prompts with their intended visual outputs. Experimental results demonstrate the superiority of our methods over baseline approaches, as evidenced in our new offline evaluation method and online tests. Our code and dataset are available at https://github.com/zzjchen/Tailored-Visions.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 27 of its 36 references that are in the map. First posted on arXiv 12 Oct 2023.

Builds on

27 of 36 references.

References · 27

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  8. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  9. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  10. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  11. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  12. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
15 more
  1. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,099
  2. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  3. 2023Large Language Models as Optimizerscited by 830
  4. 2023Baichuan 2: Open Large-scale Language Modelscited by 664
  5. 2021Large Dual Encoders Are Generalizable Retrieverscited by 454
  6. 2023InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuningcited by 343
  7. 2023Language Modeling Is Compressioncited by 225
  8. 2022Optimizing Prompts for Text-to-Image Generationcited by 204
  9. 2023Encoder-based Domain Tuning for Fast Personalization of Text-to-Image Modelscited by 203
  10. 2023Key-Locked Rank One Editing for Text-to-Image Personalizationcited by 194
  11. 2023EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizerscited by 189
  12. 2022"This is my unicorn, Fluffy": Personalizing frozen vision-language representationscited by 95
  13. 2023Query Expansion by Prompting Large Language Modelscited by 56
  14. 2023SUR-adapter: Enhancing Text-to-Image Pre-trained Diffusion Models with Large Language Modelscited by 29
  15. 2023Generative Query Reformulation for Effective Adhoc Searchcited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 13; 2026: 7.

23 papers.

Sort:

By area:

  1. 2024From Matching to Generation: A Survey on Generative Information Retrievalcited by 89
  2. 2024Controllable Generation with Text-to-Image Diffusion Models: A Surveycited by 82
  3. 2025SnapMoGen: Human Motion Generation from Expressive Textscited by 44
  4. 2025The Devil is in the Prompts: Retrieval-Augmented Prompt Optimization for Text-to-Video Generationcited by 21
  5. 2025RAPO++: Cross-Stage Prompt Optimization for Text-to-Video Generation via Data Alignment and Test-Time Scalingcited by 9
  6. 2025DRC: Enhancing Personalized Image Generation via Disentangled Representation Compositioncited by 8
  7. 2025POET: Supporting Prompting Creativity and Personalization with Automated Expansion of Text-to-Image Generationcited by 6
  8. 2025Draw Your Mind: Personalized Generation via Condition-Level Modeling in Text-to-Image Diffusion Modelscited by 5
  9. 2025Distribution-Conditional Generation: From Class Distribution to Creative Generationcited by 4
  10. 2025POET: Preference Optimization for Enhanced Text-to-Image Generationcited by 3
  11. 2025DialectGen: Benchmarking and Improving Dialect Robustness in Multimodal Generationcited by 3
  12. 2025Personalized Reward Modeling for Text-to-Image Generationcited by 3
11 more
  1. 2026Premier: Personalized Preference Modulation with Learnable User Embedding in Text-to-Image Generationcited by 2
  2. 2024Specify and Edit: Overcoming Ambiguity in Text-Based Image Editingcited by 1
  3. 2025Identifying Prompted Artist Names from Generated Imagescited by 1
  4. 2026EPIG: Emotion-Based Prompting for Personalised Image Generationcited by 1
  5. 2026PIPBench: A Profile-Inclusive Framework for Personalized Image Generation Evaluationcited by 1
  6. 2026Personalized Image Aesthetic Assessment via Preference-rich Sample Mining and Cohort Mergingcited by 1
  7. 2025Generative AI in Collaborative Academic Report Writing: Advantages, Disadvantages, and Ethical Considerationscited by 0
  8. 2025Evolve to Inspire: Novelty Search for Diverse Image Generationcited by 0
  9. 2026Visual Personalization Turing Testcited by 0
  10. 2026Personalizing Text-to-Image Generation to Individual Tastecited by 0
  11. 2026ZIPP:Zero-shot Image Personalization from Personascited by 0