# Iteratively Prompting Multimodal LLMs to Reproduce Natural and AI-Generated Images arXiv: https://arxiv.org/abs/2404.13784 ยท HTML: https://mlsanity.com/paper/2404.13784/ ## Key facts - Cited by 9 papers in the map (data through 14 Oct 2026). - Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. - Builds on 29 of its 32 references that are in the map. - First posted on arXiv 21 Apr 2024. ## Placed in - Multimodal LLMs / vision-language models / [MLLM jailbreaks and adversarial robustness](https://mlsanity.com/mllm/mllm-jailbreaks-and-adversarial-robustness/) ## Builds on (most cited first, 29 of 29) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021) - [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015) - [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022) - [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023) - [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022) - [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021) - [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022) - [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021) - [Scaling Autoregressive Models for Content-Rich Text-to-Image Generation](https://mlsanity.com/paper/2206.10789/) (2022) - [PaLI: A Jointly-Scaled Multilingual Language-Image Model](https://mlsanity.com/paper/2209.06794/) (2022) - [Generative Adversarial Text to Image Synthesis](https://mlsanity.com/paper/1605.05396/) (2016) - [CogView: Mastering Text-to-Image Generation via Transformers](https://mlsanity.com/paper/2105.13290/) (2021) - [AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks](https://mlsanity.com/paper/1711.10485/) (2017) - [StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks](https://mlsanity.com/paper/1612.03242/) (2016) - [GIT: A Generative Image-to-text Transformer for Vision and Language](https://mlsanity.com/paper/2205.14100/) (2022) - [Tree-Ring Watermarks: Fingerprints for Diffusion Images that are Invisible and Robust](https://mlsanity.com/paper/2305.20030/) (2023) - [StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual Representation Learners](https://mlsanity.com/paper/2306.00984/) (2023) - [A Recipe for Watermarking Diffusion Models](https://mlsanity.com/paper/2303.10137/) (2023) - [Generating Images from Captions with Attention](https://mlsanity.com/paper/1511.02793/) (2015) - [EditGuard: Versatile Image Watermarking for Tamper Localization and Copyright Protection](https://mlsanity.com/paper/2312.08883/) (2023) - [Effective Prompt Extraction from Language Models](https://mlsanity.com/paper/2307.06865/) (2023) - [X-LXMERT: Paint, Caption and Answer Questions with Multi-Modal Transformers](https://mlsanity.com/paper/2009.11278/) (2020) - [A Taxonomy of Prompt Modifiers for Text-To-Image Generation](https://mlsanity.com/paper/2204.13988/) (2022) - [Prompt Stealing Attacks Against Text-to-Image Generation Models](https://mlsanity.com/paper/2302.09923/) (2023) - [Prompt Stealing Attacks Against Large Language Models](https://mlsanity.com/paper/2402.12959/) (2024) - [Language Models as Black-Box Optimizers for Vision-Language Models](https://mlsanity.com/paper/2309.05950/) (2023) - [StyleT2I: Toward Compositional and High-Fidelity Text-to-Image Synthesis](https://mlsanity.com/paper/2203.15799/) (2022) - [BeautifulPrompt: Towards Automatic Prompt Engineering for Text-to-Image Synthesis](https://mlsanity.com/paper/2311.06752/) (2023) ## Built on by (most cited first, 9 of 9) - [Backdooring Bias (B2) into Stable Diffusion Models](https://mlsanity.com/paper/2406.15213/) (2024) - [Vulnerability of Text-to-Image Models to Prompt Template Stealing: A Differential Evolution Approach](https://mlsanity.com/paper/2502.14285/) (2025) - [Data-Centric AI in the Age of Large Language Models](https://mlsanity.com/paper/2406.14473/) (2024) - [PROMPTMINER: Black-Box Prompt Stealing against Text-to-Image Generative Models via Reinforcement Learning and Fuzz Optimization](https://mlsanity.com/paper/2511.22119/) (2025) - [Red-Teaming Text-to-Image Models via In-Context Experience Replay and Semantic-Preserving Prompt Rewriting](https://mlsanity.com/paper/2411.16769/) (2024) - [EDITOR: Effective and Interpretable Prompt Inversion for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2506.03067/) (2025) - [Reinforcement Learning-Based Prompt Template Stealing for Text-to-Image Models](https://mlsanity.com/paper/2510.00046/) (2025) - [Text-to-Image Models Leave Identifiable Signatures: Implications for Leaderboard Security](https://mlsanity.com/paper/2510.06525/) (2025) - [Identifying Models Behind Text-to-Image Leaderboards](https://mlsanity.com/paper/2601.09647/) (2026) ## Abstract (arXiv) With the digital imagery landscape rapidly evolving, image stocks and AI-generated image marketplaces have become central to visual media. Traditional stock images now exist alongside innovative platforms that trade in prompts for AI-generated visuals, driven by sophisticated APIs like DALL-E 3 and Midjourney. This paper studies the possibility of employing multi-modal models with enhanced visual understanding to mimic the outputs of these platforms, introducing an original attack strategy. Our method leverages fine-tuned CLIP models, a multi-label classifier, and the descriptive capabilities of GPT-4V to create prompts that generate images similar to those available in marketplaces and from premium stock image providers, yet at a markedly lower expense. In presenting this strategy, we aim to spotlight a new class of economic and security considerations within the realm of digital imagery. Our findings, supported by both automated metrics and human assessment, reveal that comparable visual content can be produced for a fraction of the prevailing market prices (0.23 - 0.27 per image), emphasizing the need for awareness and strategic discussions about the integrity of digital media in an increasingly AI-integrated landscape. Our work also contributes to the field by assembling a dataset consisting of approximately 19 million prompt-image pairs generated by the popular Midjourney platform, which we plan to release publicly.