# Trading Inference-Time Compute for Adversarial Robustness

arXiv: https://arxiv.org/abs/2501.18841 · HTML: https://mlsanity.com/paper/2501.18841/

## Key facts

- Cited by 61 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 22 of its 30 references that are in the map.
- First posted on arXiv 31 Jan 2025.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Jailbroken: How Does LLM Safety Training Fail?](https://mlsanity.com/paper/2307.02483/) (2023)
- [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023)
- [Jailbreaking Black Box Large Language Models in Twenty Queries](https://mlsanity.com/paper/2310.08419/) (2023)
- [Natural Adversarial Examples](https://mlsanity.com/paper/1907.07174/) (2019)
- [Certified Adversarial Robustness via Randomized Smoothing](https://mlsanity.com/paper/1902.02918/) (2019)
- [Baseline Defenses for Adversarial Attacks Against Aligned Language Models](https://mlsanity.com/paper/2309.00614/) (2023)
- [Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks](https://mlsanity.com/paper/2404.02151/) (2024)
- [A StrongREJECT for Empty Jailbreaks](https://mlsanity.com/paper/2402.10260/) (2024)
- [Measuring short-form factuality in large language models](https://mlsanity.com/paper/2411.04368/) (2024)
- [How Robust is Google's Bard to Adversarial Image Attacks?](https://mlsanity.com/paper/2309.11751/) (2023)
- [Tensor Trust: Interpretable Prompt Injection Attacks from an Online Game](https://mlsanity.com/paper/2311.01011/) (2023)
- [(Certified!!) Adversarial Robustness for Free!](https://mlsanity.com/paper/2206.10550/) (2022)
- [A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinking](https://mlsanity.com/paper/2302.14301/) (2023)
- [Revisiting Adversarial Training for ImageNet: Architectures, Training and Generalization across Threat Models](https://mlsanity.com/paper/2303.01870/) (2023)
- [Does Refusal Training in LLMs Generalize to the Past Tense?](https://mlsanity.com/paper/2407.11969/) (2024)
- [Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?](https://mlsanity.com/paper/2407.21792/) (2024)
- [Revisiting Adversarial Training at Scale](https://mlsanity.com/paper/2401.04727/) (2024)
- [Scaling Trends in Language Model Robustness](https://mlsanity.com/paper/2407.18213/) (2024)

## Built on by (most cited first, 30 of 61)

- [OverThink: Slowdown Attacks on Reasoning LLMs](https://mlsanity.com/paper/2502.02542/) (2025)
- [Effectively Controlling Reasoning Models through Thinking Intervention](https://mlsanity.com/paper/2503.24370/) (2025)
- [Thought Crime: Backdoors and Emergent Misalignment in Reasoning Models](https://mlsanity.com/paper/2506.13206/) (2025)
- [Safety in Large Reasoning Models: A Survey](https://mlsanity.com/paper/2504.17704/) (2025)
- [Inverse Scaling in Test-Time Compute](https://mlsanity.com/paper/2507.14417/) (2025)
- [AdaSteer: Your Aligned LLM is Inherently an Adaptive Jailbreak Defender](https://mlsanity.com/paper/2504.09466/) (2025)
- [A Closer Look at System Prompt Robustness](https://mlsanity.com/paper/2502.12197/) (2025)
- [Heimdall: test-time scaling on the generative verification](https://mlsanity.com/paper/2504.10337/) (2025)
- [A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models](https://mlsanity.com/paper/2509.03871/) (2025)
- [Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atoms](https://mlsanity.com/paper/2505.20322/) (2025)
- [Security Challenges in AI Agent Deployment: Insights from a Large Scale Public Competition](https://mlsanity.com/paper/2507.20526/) (2025)
- [Reasoning as an Adaptive Defense for Safety](https://mlsanity.com/paper/2507.00971/) (2025)
- [LLMail-Inject: A Dataset from a Realistic Adaptive Prompt Injection Challenge](https://mlsanity.com/paper/2506.09956/) (2025)
- [Steering Dialogue Dynamics for Robustness against Multi-turn Jailbreaking Attacks](https://mlsanity.com/paper/2503.00187/) (2025)
- [Forecasting Rare Language Model Behaviors](https://mlsanity.com/paper/2502.16797/) (2025)
- [Two Experts Are All You Need for Steering Thinking: Reinforcing Cognitive Effort in MoE Reasoning Models Without Additional Training](https://mlsanity.com/paper/2505.14681/) (2025)
- [Position: Adversarial ML for LLMs Is Not Making Any Progress](https://mlsanity.com/paper/2502.02260/) (2025)
- [Controllable LLM Reasoning via Sparse Autoencoder-Based Steering](https://mlsanity.com/paper/2601.03595/) (2026)
- [AgentBreeder: Mitigating the AI Safety Risks of Multi-Agent Scaffolds via Self-Improvement](https://mlsanity.com/paper/2502.00757/) (2025)
- [STACK: Adversarial Attacks on LLM Safeguard Pipelines](https://mlsanity.com/paper/2506.24068/) (2025)
- [ROC-n-reroll: How verifier imperfection affects test-time scaling](https://mlsanity.com/paper/2507.12399/) (2025)
- [Test-Time Scaling in Reasoning Models Is Not Effective for Knowledge-Intensive Tasks Yet](https://mlsanity.com/paper/2509.06861/) (2025)
- [Think Twice, Generate Once: Safeguarding by Progressive Self-Reflection](https://mlsanity.com/paper/2510.01270/) (2025)
- [Refusal Falls off a Cliff: How Safety Alignment Fails in Reasoning?](https://mlsanity.com/paper/2510.06036/) (2025)
- [AegisLLM: Scaling Agentic Systems for Self-Reflective Defense in LLM Security](https://mlsanity.com/paper/2504.20965/) (2025)
- [SecInfer: Preventing Prompt Injection via Inference-time Scaling](https://mlsanity.com/paper/2509.24967/) (2025)
- [Jailbreaking to Jailbreak](https://mlsanity.com/paper/2502.09638/) (2025)
- [Beyond Safe Answers: A Benchmark for Evaluating True Risk Awareness in Large Reasoning Models](https://mlsanity.com/paper/2505.19690/) (2025)
- [RedDebate: Safer Responses Through Multi-Agent Red Teaming Debates](https://mlsanity.com/paper/2506.11083/) (2025)
- [Reasoning Introduces New Poisoning Attacks Yet Makes Them More Complicated](https://mlsanity.com/paper/2509.05739/) (2025)

## Abstract (arXiv)

We conduct experiments on the impact of increasing inference-time compute in reasoning models (specifically OpenAI o1-preview and o1-mini) on their robustness to adversarial attacks. We find that across a variety of attacks, increased inference-time compute leads to improved robustness. In many cases (with important exceptions), the fraction of model samples where the attack succeeds tends to zero as the amount of test-time compute grows. We perform no adversarial training for the tasks we study, and we increase inference-time compute by simply allowing the models to spend more compute on reasoning, independently of the form of attack. Our results suggest that inference-time compute has the potential to improve adversarial robustness for Large Language Models. We also explore new attacks directed at reasoning models, as well as settings where inference-time compute does not improve reliability, and speculate on the reasons for these as well as ways to address them.