mlsanityData to 13 Oct 2026

Reinforcement Learning-Based Prompt Template Stealing for Text-to-Image Models

arXiv 2510.00046 ↗First posted 27 Sep 2025cs.CV cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 1 more
Abstract (arXiv)

Multimodal Large Language Models (MLLMs) have transformed text-to-image workflows, allowing designers to create novel visual concepts with unprecedented speed. This progress has given rise to a thriving prompt trading market, where curated prompts that induce trademark styles are bought and sold. Although commercially attractive, prompt trading also introduces a largely unexamined security risk: the prompts themselves can be stolen. In this paper, we expose this vulnerability and present RLStealer, a reinforcement learning based prompt inversion framework that recovers its template from only a small set of example images. RLStealer treats template stealing as a sequential decision making problem and employs multiple similarity based feedback signals as reward functions to effectively explore the prompt space. Comprehensive experiments on publicly available benchmarks demonstrate that RLStealer gets state-of-the-art performance while reducing the total attack cost to under 13% of that required by existing baselines. Our further analysis confirms that RLStealer can effectively generalize across different image styles to efficiently steal unseen prompt templates. Our study highlights an urgent security threat inherent in prompt trading and lays the groundwork for developing protective standards in the emerging MLLMs marketplace.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 16 of its 21 references that are in the map. First posted on arXiv 27 Sep 2025.

Builds on

16 of 21 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Iteratively Prompting Multimodal LLMs to Reproduce Natural and AI-Generated Images21 Apr

Other references · 15

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,093
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,588
  3. 2022Training language models to follow instructions with human feedbackcited by 15,856
  4. 2017Proximal Policy Optimization Algorithmscited by 14,592
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,322
  6. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,916
  7. 2013Playing Atari with Deep Reinforcement Learningcited by 3,221
  8. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,068
  9. 2022Diffusion Models in Vision: A Surveycited by 1,012
  10. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 726
  11. 2021Design Guidelines for Prompt Engineering Text-to-Image Generative Modelscited by 218
  12. 2024Evolutionary Computation in the Era of Large Language Model: Survey and Roadmapcited by 90
3 more
  1. 2023Prompt Stealing Attacks Against Text-to-Image Generation Modelscited by 51
  2. 2023AutoHint: Automatic Prompt Optimization with Hint Generationcited by 30
  3. 2025Vulnerability of Text-to-Image Models to Prompt Template Stealing: A Differential Evolution Approachcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025PROMPTMINER: Black-Box Prompt Stealing against Text-to-Image Generative Models via Reinforcement Learning and Fuzz Optimization
  2. 2025Towards Effective Prompt Stealing Attack against Text-to-Image Diffusion Models
  3. 2026PromptRL: Prompt Matters in RL for Flow-Based Image Generation
  4. 2025PromptEnhancer: A Simple Approach to Enhance Text-to-Image Models via Chain-of-Thought Prompt Rewriting
  5. 2025BadReward: Clean-Label Poisoning of Reward Models in Text-to-Image RLHF
  6. 2026Covert Visual Prompt Injection against Commercial Multimodal Large Language Models
  7. 2026PromptEcho: Annotation-Free Reward from Vision-Language Models for Text-to-Image Reinforcement Learning
  8. 2024Red-Teaming Text-to-Image Models via In-Context Experience Replay and Semantic-Preserving Prompt Rewriting