# Adversarial Reasoning at Jailbreaking Time

arXiv: https://arxiv.org/abs/2502.01633 · HTML: https://mlsanity.com/paper/2502.01633/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 70 of its 75 references that are in the map.
- First posted on arXiv 3 Feb 2025.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- LLM safety and security / [Jailbreaking](https://mlsanity.com/safety/jailbreaking/)

## Builds on (most cited first, 30 of 70)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [Mixtral of Experts](https://mlsanity.com/paper/2401.04088/) (2024)
- [Jailbroken: How Does LLM Safety Training Fail?](https://mlsanity.com/paper/2307.02483/) (2023)
- [HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal](https://mlsanity.com/paper/2402.04249/) (2024)
- [Jailbreaking Black Box Large Language Models in Twenty Queries](https://mlsanity.com/paper/2310.08419/) (2023)
- [OpenAI o1 System Card](https://mlsanity.com/paper/2412.16720/) (2024)
- [Code as Policies: Language Model Programs for Embodied Control](https://mlsanity.com/paper/2209.07753/) (2022)
- [Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations](https://mlsanity.com/paper/2312.06674/) (2023)
- [Red Teaming Language Models with Language Models](https://mlsanity.com/paper/2202.03286/) (2022)
- [Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations](https://mlsanity.com/paper/2312.08935/) (2023)
- [Solving math word problems with process- and outcome-based feedback](https://mlsanity.com/paper/2211.14275/) (2022)
- [Show Your Work: Scratchpads for Intermediate Computation with Language Models](https://mlsanity.com/paper/2112.00114/) (2021)
- [AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models](https://mlsanity.com/paper/2310.04451/) (2023)
- [A Survey on In-context Learning](https://mlsanity.com/paper/2301.00234/) (2022)
- [Tree of Attacks: Jailbreaking Black-Box LLMs Automatically](https://mlsanity.com/paper/2312.02119/) (2023)
- [How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs](https://mlsanity.com/paper/2401.06373/) (2024)

## Built on by (most cited first, 29 of 29)

- [Parallel-R1: Towards Parallel Thinking via Reinforcement Learning](https://mlsanity.com/paper/2509.07980/) (2025)
- [100 Days After DeepSeek-R1: A Survey on Replication Studies and More Directions for Reasoning Language Models](https://mlsanity.com/paper/2505.00551/) (2025)
- [Antidistillation Sampling](https://mlsanity.com/paper/2504.13146/) (2025)
- [Refusal Falls off a Cliff: How Safety Alignment Fails in Reasoning?](https://mlsanity.com/paper/2510.06036/) (2025)
- [Jailbreaking to Jailbreak](https://mlsanity.com/paper/2502.09638/) (2025)
- [Benchmarking Misuse Mitigation Against Covert Adversaries](https://mlsanity.com/paper/2506.06414/) (2025)
- [ARMOR: Aligning Secure and Safe Large Language Models via Meticulous Reasoning](https://mlsanity.com/paper/2507.11500/) (2025)
- [AutoRAN: Automated Hijacking of Safety Reasoning in Large Reasoning Models](https://mlsanity.com/paper/2505.10846/) (2025)
- [T-MAP: Red-Teaming LLM Agents with Trajectory-aware Evolutionary Search](https://mlsanity.com/paper/2603.22341/) (2026)
- [Jailbreak Distillation: Renewable Safety Benchmarking](https://mlsanity.com/paper/2505.22037/) (2025)
- [One Token Embedding Is Enough to Deadlock Your Large Reasoning Model](https://mlsanity.com/paper/2510.15965/) (2025)
- [EcoAlign: An Economically Rational Framework for Efficient LVLM Alignment](https://mlsanity.com/paper/2511.11301/) (2025)
- [In-Context Representation Hijacking](https://mlsanity.com/paper/2512.03771/) (2025)
- [AgenticRed: Evolving Agentic Systems for Red-Teaming](https://mlsanity.com/paper/2601.13518/) (2026)
- [Noise Injection Systemically Degrades Large Language Model Safety Guardrails](https://mlsanity.com/paper/2505.13500/) (2025)
- [Visual Exclusivity Attacks: Automatic Multimodal Red Teaming via Agentic Planning](https://mlsanity.com/paper/2603.20198/) (2026)
- [AutoDAN-Reasoning: Enhancing Strategies Exploration based Jailbreak Attacks with Test-Time Scaling](https://mlsanity.com/paper/2510.05379/) (2025)
- [Do Reasoning LLMs Refuse What They Infer in Long Contexts?](https://mlsanity.com/paper/2602.08874/) (2026)
- [Intent Laundering: AI Safety Datasets Are Not What They Seem](https://mlsanity.com/paper/2602.16729/) (2026)
- [MaMa: A Game-Theoretic Approach for Designing Safe Agentic Systems](https://mlsanity.com/paper/2602.04431/) (2026)
- [Contrastive Reasoning Alignment: Reinforcement Learning from Hidden Representations](https://mlsanity.com/paper/2603.17305/) (2026)
- [Understanding and Improving Continuous Adversarial Training for LLMs via In-context Learning Theory](https://mlsanity.com/paper/2604.12817/) (2026)
- [FAPO: Fully Automated Prompt Optimization of Multi-Step LLM Pipelines](https://mlsanity.com/paper/2606.19605/) (2026)
- [CONTRA: Red-Teaming Configurations of Personalizable Agents](https://mlsanity.com/paper/2607.03220/) (2026)
- [Safe2Harm: Semantic Isomorphism Attacks for Jailbreaking Large Language Models](https://mlsanity.com/paper/2512.13703/) (2025)
- [Jailbreaking LLMs via Calibration](https://mlsanity.com/paper/2602.00619/) (2026)
- [Adversarial Reframing: A Framework for Targeted Generation in Language Models](https://mlsanity.com/paper/2605.21674/) (2026)
- [Statistical Inference for Adversarial Training: Central Limit Theorems via Optimal Transport](https://mlsanity.com/paper/2609.22240/) (2026)
- [Exploring Jailbreak Vulnerability Across Depth-Based Dynamic Inference in Large Language Models](https://mlsanity.com/paper/2610.15207/) (2026)

## Abstract (arXiv)

As large language models (LLMs) are becoming more capable and widespread, the study of their failure cases is becoming increasingly important. Recent advances in standardizing, measuring, and scaling test-time compute suggest new methodologies for optimizing models to achieve high performance on hard tasks. In this paper, we apply these advances to the task of model jailbreaking: eliciting harmful responses from aligned LLMs. We develop an adversarial reasoning approach to automatic jailbreaking that leverages a loss signal to guide the test-time compute, achieving SOTA attack success rates against many aligned LLMs, even those that aim to trade inference-time compute for adversarial robustness. Our approach introduces a new paradigm in understanding LLM vulnerabilities, laying the foundation for the development of more robust and trustworthy AI systems.