mlsanityData to 14 Oct 2026

TIPO: Text to Image with Text Presampling for Prompt Optimization

arXiv 2411.08127 ↗First posted 12 Nov 2024cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

TIPO (Text-to-Image Prompt Optimization) introduces an efficient approach for automatic prompt refinement in text-to-image (T2I) generation. Starting from simple user prompts, TIPO leverages a lightweight pre-trained model to expand these prompts into richer and more detailed versions. Conceptually, TIPO samples refined prompts from a targeted sub-distribution within the broader semantic space, preserving the original intent while significantly improving visual quality, coherence, and detail. Unlike resource-intensive methods based on large language models (LLMs) or reinforcement learning (RL), TIPO offers strong computational efficiency and scalability, opening new possibilities for effective automated prompt engineering in T2I tasks. Extensive experiments across multiple domains demonstrate that TIPO achieves stronger text alignment, reduced visual artifacts, and consistently higher human preference rates, while maintaining competitive aesthetic quality. These results highlight the effectiveness of distribution-aligned prompt engineering and point toward broader opportunities for scalable, automated refinement in text-to-image generation.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 41 of its 82 references that are in the map. First posted on arXiv 12 Nov 2024.

Builds on

41 of 82 references.

References · 41

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  6. 2023Visual Instruction Tuningcited by 7,869
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  9. 2022Flow Matching for Generative Modelingcited by 5,702
  10. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  11. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  12. 2021Zero-Shot Text-to-Image Generationcited by 4,288
29 more
  1. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  2. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  3. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  4. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
  5. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
  6. 2022Building Normalizing Flows with Stochastic Interpolantscited by 995
  7. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 864
  8. 2023Florence-2: Advancing a Unified Representation for a Variety of Vision Taskscited by 410
  9. 2024SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformerscited by 340
  10. 2024PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generationcited by 329
  11. 2022The Vendi Score: A Diversity Evaluation Metric for Machine Learningcited by 311
  12. 2024Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understandingcited by 298
  13. 2024Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillationcited by 279
  14. 2022Optimizing Prompts for Text-to-Image Generationcited by 207
  15. 2023Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion modelscited by 205
  16. 2024jina-embeddings-v3: Multilingual Embeddings With Task LoRAcited by 141
  17. 2024NVLM: Open Frontier-Class Multimodal LLMscited by 125
  18. 2017Latent Dirichlet Allocation (LDA) and Topic modeling: models, applications, a surveycited by 91
  19. 2024Improving Text-to-Image Consistency via Automatic Prompt Optimizationcited by 90
  20. 2020Improving Image Captioning with Better Use of Captionscited by 84
  21. 2024CogView3: Finer and Faster Text-to-Image Generation via Relay Diffusioncited by 72
  22. 2024Towards Understanding the Working Mechanism of Text-to-Image Diffusion Modelcited by 48
  23. 2024Dynamic Prompt Optimizing for Text-to-Image Generationcited by 45
  24. 2023Language Models as Black-Box Optimizers for Vision-Language Modelscited by 36
  25. 2024Automated Black-box Prompt Engineering for Personalized Text-to-Image Generationcited by 12
  26. 2024Leveraging Long-Context Large Language Models for Multi-Document Understanding and Summarization in Enterprise Applicationscited by 10
  27. 2024Graph-Based Captioning: Enhancing Visual Descriptions by Interconnecting Region Captionscited by 7
  28. 2024Illustrious: an Open Advanced Illustration Modelcited by 5
  29. 2024Improvements to SDXL in NovelAI Diffusion V3cited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 5.

6 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025POET: Preference Optimization for Enhanced Text-to-Image Generation
  2. 2025Test-time Prompt Refinement for Text-to-Image Models
  3. 2025Free Lunch Alignment of Text-to-Image Diffusion Models without Preference Image Pairs
  4. 2024PromptCharm: Text-to-Image Generation through Multi-modal Prompting and Refinement
  5. 2025Self-Rewarding Large Vision-Language Models for Optimizing Prompts in Text-to-Image Generation
  6. 2025Reward-Agnostic Prompt Optimization for Text-to-Image Diffusion Models
  7. 2024Batch-Instructed Gradient for Prompt Evolution:Systematic Prompt Optimization for Enhanced Text-to-Image Synthesis
  8. 2025PromptEnhancer: A Simple Approach to Enhance Text-to-Image Models via Chain-of-Thought Prompt Rewriting