mlsanityData to 7 Oct 2026

Escaping Alignment: A Physical Trap Model of Best-of-N Jailbreaking

arXiv 2609.32116 ↗First posted 26 Sep 2026cs.AI cond-mat.stat-mechcited by 0

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Best-of-N jailbreaking (BoN) bypasses safeguards of aligned models by drawing N independent augmentations of an unsafe prompt and sampling M completions of each. Previous works have shown that the attack success rate (ASR) seems to follow a power-law in N, which we challenge. The exponent drifts with N, with an exponential crossover which is a finite-size artifact of the adversarial dataset. Little work has been done to explore the entire two-budget (N, M) attack surface as well as its dependence on the generation temperature T. We introduce a simple barrier model where each prompt has a baseline safety level and each augmentation a random thermally activated barrier. Then four numbers, each backed by an interpretable safety mechanism, determine the entire (N, M) attack surface. They extrapolate predictions from N ≤ 100 to N = 104, collapse five distinct models on the same scaling function and predict ASR at different temperatures from the one they were fitted at.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 27 of its 34 references that are in the map. First posted on arXiv 26 Sep 2026.

Builds on

27 of 34 references.

Closest ancestors · 12

Same direction, newest first

  1. 2026Jailbreak Scaling Laws for Large Language Models: Polynomial-Exponential Crossover11 Mar
  2. 2026Statistical Estimation of Adversarial Risk in Large Language Models under Best-of-N Sampling30 Jan
  3. 2025How Do Large Language Monkeys Get Their Power (Laws)?24 Feb
  4. 2024Best-of-N Jailbreaking4 Dec
  5. 2024JailbreakEval: An Integrated Toolkit for Evaluating Jailbreak Attempts Against Large Language Models13 Jun
  6. 2024A StrongREJECT for Empty Jailbreaks15 Feb
  7. 2024Attacking Large Language Models with Projected Gradient Descent14 Feb
  8. 2024Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Space14 Feb
  9. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  10. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks5 Oct
  11. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models3 Oct
  12. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul

Other references · 15

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2024Qwen2.5 Technical Reportcited by 4,713
  5. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  6. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  7. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  8. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  9. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  10. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
  11. 2025The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independencecited by 104
  12. 2021Learning Curve Theorycited by 79
3 more
  1. 2024A Simple Model of Inference Scaling Lawscited by 36
  2. 2025Efficient Prediction of Pass@k Scaling in Large Language Modelscited by 14
  3. 2026Learning Shrinks the Hard Tail: Training-Dependent Inference Scaling in a Solvable Linear Modelcited by 6

Built on by

Not cited yet.