# Defense Against the Dark Prompts: Mitigating Best-of-N Jailbreaking with Prompt Evaluation

arXiv: https://arxiv.org/abs/2502.00580 · HTML: https://mlsanity.com/paper/2502.00580/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Jailbreaking, one of 12 directions in LLM safety and security.
- First posted on arXiv 1 Feb 2025.

## Placed in

- LLM safety and security / [Jailbreaking](https://mlsanity.com/safety/jailbreaking/)

## Built on by (most cited first, 8 of 8)

- [Intelligent AI Delegation](https://mlsanity.com/paper/2602.11865/) (2026)
- [Distributional AGI Safety](https://mlsanity.com/paper/2512.16856/) (2025)
- [Beyond Surface-Level Patterns: An Essence-Driven Defense Framework Against Jailbreak Attacks in LLMs](https://mlsanity.com/paper/2502.19041/) (2025)
- [JailNewsBench: Multi-Lingual and Regional Benchmark for Fake News Generation under Jailbreak Attacks](https://mlsanity.com/paper/2603.01291/) (2026)
- [CCFC: Core & Core-Full-Core Dual-Track Defense for LLM Jailbreak Protection](https://mlsanity.com/paper/2508.14128/) (2025)
- [PD3F: A Pluggable and Dynamic DoS-Defense Framework Against Resource Consumption Attacks Targeting Large Language Models](https://mlsanity.com/paper/2505.18680/) (2025)
- [AI Chaperones Are (Really) All You Need to Prevent Parasocial Relationships with Chatbots](https://mlsanity.com/paper/2508.15748/) (2025)
- [Depth, Not Breadth: Best-of-N Jailbreaking Beyond Surface Noise](https://mlsanity.com/paper/2607.26639/) (2026)

## Abstract (arXiv)

Recent work showed Best-of-N (BoN) jailbreaking using repeated use of random augmentations (such as capitalization, punctuation, etc) is effective against all major large language models (LLMs). We have found that 100% of the BoN paper's successful jailbreaks (confidence interval [99.65%, 100.00%]) and 99.8% of successful jailbreaks in our replication (confidence interval [99.28%, 99.98%]) were blocked with our Defense Against The Dark Prompts (DATDP) method. The DATDP algorithm works by repeatedly utilizing an evaluation LLM to evaluate a prompt for dangerous or manipulative behaviors--unlike some other approaches, DATDP also explicitly looks for jailbreaking attempts--until a robust safety rating is generated. This success persisted even when utilizing smaller LLMs to power the evaluation (Claude and LLaMa-3-8B-instruct proved almost equally capable). These results show that, though language models are sensitive to seemingly innocuous changes to inputs, they seem also capable of successfully evaluating the dangers of these inputs. Versions of DATDP can therefore be added cheaply to generative AI systems to produce an immediate significant increase in safety.