mlsanityData to 7 Oct 2026

Adversarial Reasoning at Jailbreaking Time

arXiv 2502.01633 ↗First posted 3 Feb 2025cs.LG cs.AIcited by 27

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

As large language models (LLMs) are becoming more capable and widespread, the study of their failure cases is becoming increasingly important. Recent advances in standardizing, measuring, and scaling test-time compute suggest new methodologies for optimizing models to achieve high performance on hard tasks. In this paper, we apply these advances to the task of model jailbreaking: eliciting harmful responses from aligned LLMs. We develop an adversarial reasoning approach to automatic jailbreaking that leverages a loss signal to guide the test-time compute, achieving SOTA attack success rates against many aligned LLMs, even those that aim to trade inference-time compute for adversarial robustness. Our approach introduces a new paradigm in understanding LLM vulnerabilities, laying the foundation for the development of more robust and trustworthy AI systems.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 70 of its 75 references that are in the map. First posted on arXiv 3 Feb 2025.

Builds on

70 of 75 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024OpenAI o1 System Card21 Dec
  2. 2024Deliberative Alignment: Reasoning Enables Safer Language Models20 Dec
  3. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs12 Jan

Other references · 67

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  8. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  9. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  10. 2017Deep reinforcement learning from human preferencescited by 3,846
  11. 2023Let's Verify Step by Stepcited by 3,598
  12. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
38 more
  1. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  2. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  3. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  4. 2024Mixtral of Expertscited by 1,785
  5. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  6. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  7. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  8. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  9. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  10. 2022Red Teaming Language Models with Language Modelscited by 983
  11. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 895
  12. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  13. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  14. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
  15. 2022A Survey on In-context Learningcited by 717
  16. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 619
  17. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 494
  18. 2024Generative Verifiers: Reward Modeling as Next-Token Predictioncited by 465
  19. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 450
  20. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
  21. 2023Detecting Language Model Attacks with Perplexitycited by 411
  22. 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agentscited by 406
  23. 2024A StrongREJECT for Empty Jailbreakscited by 352
  24. 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 347
  25. 2023Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discoverycited by 329
  26. 2024Improving Alignment and Robustness with Circuit Breakerscited by 302
  27. 2024Large Language Models for Mathematical Reasoning: Progresses and Challengescited by 283
  28. 2024TextGrad: Automatic "Differentiation" via Textcited by 250
  29. 2020AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Promptscited by 246
  30. 2024Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learningcited by 238
  31. 2022Exploring Length Generalization in Large Language Modelscited by 204
  32. 2024Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Promptscited by 189
  33. 2024AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMscited by 185
  34. 2023Language-Driven Representation Learning for Roboticscited by 163
  35. 2023MART: Improving LLM Safety with Multi-round Automatic Red-Teamingcited by 161
  36. 2024AmpleGCG: Learning a Universal and Transferable Generative Model of Adversarial Suffixes for Jailbreaking Both Open and Closed LLMscited by 160
  37. 2024Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMscited by 158
  38. 2024AdvPrompter: Fast Adaptive Adversarial Prompting for LLMscited by 146

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 15; 2026: 12.

27 papers.

Sort:

By area:

  1. 2025Parallel-R1: Towards Parallel Thinking via Reinforcement Learningcited by 59
  2. 2025100 Days After DeepSeek-R1: A Survey on Replication Studies and More Directions for Reasoning Language Modelscited by 29
  3. 2025Antidistillation Samplingcited by 24
  4. 2025Refusal Falls off a Cliff: How Safety Alignment Fails in Reasoning?cited by 11
  5. 2025Jailbreaking to Jailbreakcited by 8
  6. 2025Benchmarking Misuse Mitigation Against Covert Adversariescited by 8
  7. 2025ARMOR: Aligning Secure and Safe Large Language Models via Meticulous Reasoningcited by 8
  8. 2025AutoRAN: Automated Hijacking of Safety Reasoning in Large Reasoning Modelscited by 7
  9. 2026T-MAP: Red-Teaming LLM Agents with Trajectory-aware Evolutionary Searchcited by 6
  10. 2025EcoAlign: An Economically Rational Framework for Efficient LVLM Alignmentcited by 5
  11. 2025In-Context Representation Hijackingcited by 5
  12. 2025Noise Injection Systemically Degrades Large Language Model Safety Guardrailscited by 4
15 more
  1. 2025Jailbreak Distillation: Renewable Safety Benchmarkingcited by 4
  2. 2025One Token Embedding Is Enough to Deadlock Your Large Reasoning Modelcited by 4
  3. 2026AgenticRed: Evolving Agentic Systems for Red-Teamingcited by 4
  4. 2026Visual Exclusivity Attacks: Automatic Multimodal Red Teaming via Agentic Planningcited by 4
  5. 2025AutoDAN-Reasoning: Enhancing Strategies Exploration based Jailbreak Attacks with Test-Time Scalingcited by 3
  6. 2026Do Reasoning LLMs Refuse What They Infer in Long Contexts?cited by 3
  7. 2026MaMa: A Game-Theoretic Approach for Designing Safe Agentic Systemscited by 2
  8. 2026Contrastive Reasoning Alignment: Reinforcement Learning from Hidden Representationscited by 2
  9. 2026Intent Laundering: AI Safety Datasets Are Not What They Seemcited by 1
  10. 2026Understanding and Improving Continuous Adversarial Training for LLMs via In-context Learning Theorycited by 1
  11. 2026FAPO: Fully Automated Prompt Optimization of Multi-Step LLM Pipelinescited by 1
  12. 2025Safe2Harm: Semantic Isomorphism Attacks for Jailbreaking Large Language Modelscited by 0
  13. 2026Jailbreaking LLMs via Calibrationcited by 0
  14. 2026CONTRA: Red-Teaming Configurations of Personalizable Agentscited by 0
  15. 2026Statistical Inference for Adversarial Training: Central Limit Theorems via Optimal Transportcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Survey
  2. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks
  3. 2026Jailbreaking Leaves a Trace: Understanding and Detecting Jailbreak Attacks from Internal Representations of Large Language Models
  4. 2024Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editing
  5. 2025ForgeDAN: An Evolutionary Framework for Jailbreaking Aligned Large Language Models
  6. 2024Reasoning as a Weapon: Adaptive Dual-Path Jailbreak Attack on Large Language Models
  7. 2024Defending against Jailbreak through Early Exit Generation of Large Language Models
  8. 2025Scaling Patterns in Adversarial Alignment: Evidence from Multi-LLM Jailbreak Experiments