# Tailored Visions: Enhancing Text-to-Image Generation with Personalized Prompt Rewriting

arXiv: https://arxiv.org/abs/2310.08129 · HTML: https://mlsanity.com/paper/2310.08129/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 36 references that are in the map.
- First posted on arXiv 12 Oct 2023.

## Builds on (most cited first, 27 of 27)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [Multi-Concept Customization of Text-to-Image Diffusion](https://mlsanity.com/paper/2212.04488/) (2022)
- [Large Language Models Are Human-Level Prompt Engineers](https://mlsanity.com/paper/2211.01910/) (2022)
- [Large Language Models as Optimizers](https://mlsanity.com/paper/2309.03409/) (2023)
- [Baichuan 2: Open Large-scale Language Models](https://mlsanity.com/paper/2309.10305/) (2023)
- [Large Dual Encoders Are Generalizable Retrievers](https://mlsanity.com/paper/2112.07899/) (2021)
- [InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuning](https://mlsanity.com/paper/2304.03411/) (2023)
- [Language Modeling Is Compression](https://mlsanity.com/paper/2309.10668/) (2023)
- [Optimizing Prompts for Text-to-Image Generation](https://mlsanity.com/paper/2212.09611/) (2022)
- [Encoder-based Domain Tuning for Fast Personalization of Text-to-Image Models](https://mlsanity.com/paper/2302.12228/) (2023)
- [Key-Locked Rank One Editing for Text-to-Image Personalization](https://mlsanity.com/paper/2305.01644/) (2023)
- [EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizers](https://mlsanity.com/paper/2309.08532/) (2023)
- ["This is my unicorn, Fluffy": Personalizing frozen vision-language representations](https://mlsanity.com/paper/2204.01694/) (2022)
- [Query Expansion by Prompting Large Language Models](https://mlsanity.com/paper/2305.03653/) (2023)
- [SUR-adapter: Enhancing Text-to-Image Pre-trained Diffusion Models with Large Language Models](https://mlsanity.com/paper/2305.05189/) (2023)
- [Generative Query Reformulation for Effective Adhoc Search](https://mlsanity.com/paper/2308.00415/) (2023)

## Built on by (most cited first, 23 of 23)

- [From Matching to Generation: A Survey on Generative Information Retrieval](https://mlsanity.com/paper/2404.14851/) (2024)
- [Controllable Generation with Text-to-Image Diffusion Models: A Survey](https://mlsanity.com/paper/2403.04279/) (2024)
- [SnapMoGen: Human Motion Generation from Expressive Texts](https://mlsanity.com/paper/2507.09122/) (2025)
- [The Devil is in the Prompts: Retrieval-Augmented Prompt Optimization for Text-to-Video Generation](https://mlsanity.com/paper/2504.11739/) (2025)
- [RAPO++: Cross-Stage Prompt Optimization for Text-to-Video Generation via Data Alignment and Test-Time Scaling](https://mlsanity.com/paper/2510.20206/) (2025)
- [DRC: Enhancing Personalized Image Generation via Disentangled Representation Composition](https://mlsanity.com/paper/2504.17349/) (2025)
- [POET: Supporting Prompting Creativity and Personalization with Automated Expansion of Text-to-Image Generation](https://mlsanity.com/paper/2504.13392/) (2025)
- [Draw Your Mind: Personalized Generation via Condition-Level Modeling in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2508.03481/) (2025)
- [Distribution-Conditional Generation: From Class Distribution to Creative Generation](https://mlsanity.com/paper/2505.03667/) (2025)
- [POET: Preference Optimization for Enhanced Text-to-Image Generation](https://mlsanity.com/paper/2510.12041/) (2025)
- [Personalized Reward Modeling for Text-to-Image Generation](https://mlsanity.com/paper/2511.19458/) (2025)
- [DialectGen: Benchmarking and Improving Dialect Robustness in Multimodal Generation](https://mlsanity.com/paper/2510.14949/) (2025)
- [Premier: Personalized Preference Modulation with Learnable User Embedding in Text-to-Image Generation](https://mlsanity.com/paper/2603.20725/) (2026)
- [Specify and Edit: Overcoming Ambiguity in Text-Based Image Editing](https://mlsanity.com/paper/2407.20232/) (2024)
- [Identifying Prompted Artist Names from Generated Images](https://mlsanity.com/paper/2507.18633/) (2025)
- [Evolve to Inspire: Novelty Search for Diverse Image Generation](https://mlsanity.com/paper/2511.00686/) (2025)
- [EPIG: Emotion-Based Prompting for Personalised Image Generation](https://mlsanity.com/paper/2606.13247/) (2026)
- [PIPBench: A Profile-Inclusive Framework for Personalized Image Generation Evaluation](https://mlsanity.com/paper/2607.06440/) (2026)
- [Personalized Image Aesthetic Assessment via Preference-rich Sample Mining and Cohort Merging](https://mlsanity.com/paper/2607.15752/) (2026)
- [Generative AI in Collaborative Academic Report Writing: Advantages, Disadvantages, and Ethical Considerations](https://mlsanity.com/paper/2504.08832/) (2025)
- [Visual Personalization Turing Test](https://mlsanity.com/paper/2601.22680/) (2026)
- [Personalizing Text-to-Image Generation to Individual Taste](https://mlsanity.com/paper/2604.07427/) (2026)
- [ZIPP:Zero-shot Image Personalization from Personas](https://mlsanity.com/paper/2606.08841/) (2026)

## Abstract (arXiv)

Despite significant progress in the field, it is still challenging to create personalized visual representations that align closely with the desires and preferences of individual users. This process requires users to articulate their ideas in words that are both comprehensible to the models and accurately capture their vision, posing difficulties for many users. In this paper, we tackle this challenge by leveraging historical user interactions with the system to enhance user prompts. We propose a novel approach that involves rewriting user prompts based on a newly collected large-scale text-to-image dataset with over 300k prompts from 3115 users. Our rewriting model enhances the expressiveness and alignment of user prompts with their intended visual outputs. Experimental results demonstrate the superiority of our methods over baseline approaches, as evidenced in our new offline evaluation method and online tests. Our code and dataset are available at https://github.com/zzjchen/Tailored-Visions.