mlsanityData to 7 Oct 2026

Position: Towards Implicit Prompt For Text-To-Image Models

arXiv 2403.02118 ↗First posted 4 Mar 2024cs.CY cs.AI cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

Recent text-to-image (T2I) models have had great success, and many benchmarks have been proposed to evaluate their performance and safety. However, they only consider explicit prompts while neglecting implicit prompts (hint at a target without explicitly mentioning it). These prompts may get rid of safety constraints and pose potential threats to the applications of these models. This position paper highlights the current state of T2I models toward implicit prompts. We present a benchmark named ImplicitBench and conduct an investigation on the performance and impacts of implicit prompts with popular T2I models. Specifically, we design and collect more than 2,000 implicit prompts of three aspects: General Symbols, Celebrity Privacy, and Not-Safe-For-Work (NSFW) Issues, and evaluate six well-known T2I models' capabilities under these implicit prompts. Experiment results show that (1) T2I models are able to accurately create various target symbols indicated by implicit prompts; (2) Implicit prompts bring potential risks of privacy leakage for T2I models. (3) Constraints of NSFW in most of the evaluated T2I models can be bypassed with implicit prompts. We call for increased attention to the potential and risks of implicit prompts in the T2I community and further investigation into the capabilities and impacts of implicit prompts, advocating for a balanced approach that harnesses their benefits while mitigating their risks.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 35 of its 48 references that are in the map. First posted on arXiv 4 Mar 2024.

Builds on

35 of 48 references.

References · 35

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2023Visual Instruction Tuningcited by 7,792
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  9. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  10. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  11. 2016Improved Techniques for Training GANscited by 2,938
  12. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
23 more
  1. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  2. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
  3. 2016Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networkscited by 895
  4. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 719
  5. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  6. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  7. 2017Generative Adversarial Networks: An Overviewcited by 588
  8. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 501
  9. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  10. 2022CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformerscited by 305
  11. 2023SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Modelscited by 283
  12. 2022Red-Teaming the Stable Diffusion Safety Filtercited by 257
  13. 2023T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generationcited by 253
  14. 2023Stable Bias: Analyzing Societal Representations in Diffusion Modelscited by 225
  15. 2022Optimizing Prompts for Text-to-Image Generationcited by 204
  16. 2023Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Modelscited by 171
  17. 2014Deep Learning Face Representation by Joint Identification-Verificationcited by 162
  18. 2020DF-GAN: A Simple and Effective Baseline for Text-to-Image Synthesiscited by 161
  19. 2023Holistic Evaluation of Text-To-Image Modelscited by 160
  20. 2023HRS-Bench: Holistic, Reliable and Scalable Benchmark for Text-to-Image Modelscited by 94
  21. 2023The Bias Amplification Paradox in Text-to-Image Generationcited by 69
  22. 2022Human Evaluation of Text-to-Image Models on a Multi-Task Benchmarkcited by 41
  23. 2023MultiFusion: Fusing Pre-Trained Models for Multi-Lingual, Multi-Modal Image Generationcited by 14

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.