# GenAI Security: Outsmarting the Bots with a Proactive Testing Framework

arXiv: https://arxiv.org/abs/2505.18172 · HTML: https://mlsanity.com/paper/2505.18172/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 6 references that are in the map.
- First posted on arXiv 14 May 2025.

## Builds on (most cited first, 5 of 5)

- [SneakyPrompt: Jailbreaking Text-to-image Generative Models](https://mlsanity.com/paper/2305.12082/) (2023)
- [Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs](https://mlsanity.com/paper/2307.10490/) (2023)
- [Red-Teaming for Generative AI: Silver Bullet or Security Theater?](https://mlsanity.com/paper/2401.15897/) (2024)
- [SurrogatePrompt: Bypassing the Safety Filter of Text-to-Image Models via Substitution](https://mlsanity.com/paper/2309.14122/) (2023)
- [SPML: A DSL for Defending Language Models Against Prompt Attacks](https://mlsanity.com/paper/2402.11755/) (2024)

## Abstract (arXiv)

The increasing sophistication and integration of Generative AI (GenAI) models into diverse applications introduce new security challenges that traditional methods struggle to address. This research explores the critical need for proactive security measures to mitigate the risks associated with malicious exploitation of GenAI systems. We present a framework encompassing key approaches, tools, and strategies designed to outmaneuver even advanced adversarial attacks, emphasizing the importance of securing GenAI innovation against potential liabilities. We also empirically prove the effectiveness of the said framework by testing it against the SPML Chatbot Prompt Injection Dataset. This work highlights the shift from reactive to proactive security practices essential for the safe and responsible deployment of GenAI technologies