mlsanityData to 7 Oct 2026

SSP: A Simple and Safe automatic Prompt engineering method towards realistic image synthesis on LVM

arXiv 2401.01128 ↗First posted 2 Jan 2024cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Recently, text-to-image (T2I) synthesis has undergone significant advancements, particularly with the emergence of Large Language Models (LLM) and their enhancement in Large Vision Models (LVM), greatly enhancing the instruction-following capabilities of traditional T2I models. Nevertheless, previous methods focus on improving generation quality but introduce unsafe factors into prompts. We explore that appending specific camera descriptions to prompts can enhance safety performance. Consequently, we propose a simple and safe prompt engineering method (SSP) to improve image generation quality by providing optimal camera descriptions. Specifically, we create a dataset from multi-datasets as original prompts. To select the optimal camera, we design an optimal camera matching approach and implement a classifier for original prompts capable of automatically matching. Appending camera descriptions to original prompts generates optimized prompts for further LVM image generation. Experiments demonstrate that SSP improves semantic consistency by an average of 16% compared to others and safety metrics by 48.9%.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 21 of its 31 references that are in the map. First posted on arXiv 2 Jan 2024.

Builds on

21 of 31 references.

References · 21

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  8. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  9. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  10. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  11. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  12. 2019How Can We Know What Language Models Know?cited by 953
9 more
  1. 2016Generative Adversarial Text to Image Synthesiscited by 735
  2. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  3. 2022Latent Video Diffusion Models for High-Fidelity Long Video Generationcited by 385
  4. 2022DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Modelscited by 376
  5. 2022CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformerscited by 305
  6. 2023PromptMagician: Interactive Prompt Engineering for Text-to-Image Creationcited by 58
  7. 2022A Taxonomy of Prompt Modifiers for Text-To-Image Generationcited by 54
  8. 2022Best Prompts for Text-to-Image Models and How to Find Themcited by 35
  9. 2023SurrogatePrompt: Bypassing the Safety Filter of Text-to-Image Models via Substitutioncited by 32

Built on by

Not cited yet.