# Prompt Optimization and Evaluation for LLM Automated Red Teaming

arXiv: https://arxiv.org/abs/2507.22133 · HTML: https://mlsanity.com/paper/2507.22133/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security.
- Builds on 2 of its 8 references that are in the map.
- First posted on arXiv 29 Jul 2025.

## Placed in

- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Builds on (most cited first, 2 of 2)

- [Red Teaming Language Models with Language Models](https://mlsanity.com/paper/2202.03286/) (2022)
- [Attacks, Defenses and Evaluations for LLM Conversation Safety: A Survey](https://mlsanity.com/paper/2402.09283/) (2024)

## Built on by (most cited first, 1 of 1)

- [Beyond "I cannot fulfill this request": Alleviating Rigid Rejection in LLMs via Label Enhancement](https://mlsanity.com/paper/2605.07883/) (2026)

## Abstract (arXiv)

Applications that use Large Language Models (LLMs) are becoming widespread, making the identification of system vulnerabilities increasingly important. Automated Red Teaming accelerates this effort by using an LLM to generate and execute attacks against target systems. Attack generators are evaluated using the Attack Success Rate (ASR) the sample mean calculated over the judgment of success for each attack. In this paper, we introduce a method for optimizing attack generator prompts that applies ASR to individual attacks. By repeating each attack multiple times against a randomly seeded target, we measure an attack's discoverability the expectation of the individual attack success. This approach reveals exploitable patterns that inform prompt optimization, ultimately enabling more robust evaluation and refinement of generators.