mlsanityData to 7 Oct 2026

Trading Inference-Time Compute for Adversarial Robustness

arXiv 2501.18841 ↗First posted 31 Jan 2025cs.LG cs.CRcited by 60

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

We conduct experiments on the impact of increasing inference-time compute in reasoning models (specifically OpenAI o1-preview and o1-mini) on their robustness to adversarial attacks. We find that across a variety of attacks, increased inference-time compute leads to improved robustness. In many cases (with important exceptions), the fraction of model samples where the attack succeeds tends to zero as the amount of test-time compute grows. We perform no adversarial training for the tasks we study, and we increase inference-time compute by simply allowing the models to spend more compute on reasoning, independently of the form of attack. Our results suggest that inference-time compute has the potential to improve adversarial robustness for Large Language Models. We also explore new attacks directed at reasoning models, as well as settings where inference-time compute does not improve reliability, and speculate on the reasons for these as well as ways to address them.

Key facts. Cited by 60 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 22 of its 30 references that are in the map. First posted on arXiv 31 Jan 2025.

Builds on

22 of 30 references.

References · 22

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  5. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  6. 2019Natural Adversarial Examplescited by 1,401
  7. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  8. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  9. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  10. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  11. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 450
  12. 2024A StrongREJECT for Empty Jailbreakscited by 352
10 more
  1. 2024Measuring short-form factuality in large language modelscited by 345
  2. 2023How Robust is Google's Bard to Adversarial Image Attacks?cited by 177
  3. 2023Tensor Trust: Interpretable Prompt Injection Attacks from an Online Gamecited by 135
  4. 2022(Certified!!) Adversarial Robustness for Free!cited by 130
  5. 2023A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinkingcited by 85
  6. 2023Revisiting Adversarial Training for ImageNet: Architectures, Training and Generalization across Threat Modelscited by 79
  7. 2024Does Refusal Training in LLMs Generalize to the Past Tense?cited by 74
  8. 2024Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?cited by 56
  9. 2024Revisiting Adversarial Training at Scalecited by 33
  10. 2024Scaling Trends in Language Model Robustnesscited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 49; 2026: 11.

60 papers.

Sort:

By area:

  1. 2025OverThink: Slowdown Attacks on Reasoning LLMscited by 94
  2. 2025Effectively Controlling Reasoning Models through Thinking Interventioncited by 63
  3. 2025Safety in Large Reasoning Models: A Surveycited by 58
  4. 2025Thought Crime: Backdoors and Emergent Misalignment in Reasoning Modelscited by 57
  5. 2025Inverse Scaling in Test-Time Computecited by 49
  6. 2025AdaSteer: Your Aligned LLM is Inherently an Adaptive Jailbreak Defendercited by 37
  7. 2025A Closer Look at System Prompt Robustnesscited by 31
  8. 2025Heimdall: test-time scaling on the generative verificationcited by 26
  9. 2025Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atomscited by 23
  10. 2025Reasoning as an Adaptive Defense for Safetycited by 22
  11. 2025Security Challenges in AI Agent Deployment: Insights from a Large Scale Public Competitioncited by 22
  12. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 22
38 more
  1. 2025Steering Dialogue Dynamics for Robustness against Multi-turn Jailbreaking Attackscited by 20
  2. 2025LLMail-Inject: A Dataset from a Realistic Adaptive Prompt Injection Challengecited by 20
  3. 2025Forecasting Rare Language Model Behaviorscited by 16
  4. 2025Two Experts Are All You Need for Steering Thinking: Reinforcing Cognitive Effort in MoE Reasoning Models Without Additional Trainingcited by 16
  5. 2025Position: Adversarial ML for LLMs Is Not Making Any Progresscited by 14
  6. 2025STACK: Adversarial Attacks on LLM Safeguard Pipelinescited by 14
  7. 2025AgentBreeder: Mitigating the AI Safety Risks of Multi-Agent Scaffolds via Self-Improvementcited by 13
  8. 2026Controllable LLM Reasoning via Sparse Autoencoder-Based Steeringcited by 12
  9. 2025ROC-n-reroll: How verifier imperfection affects test-time scalingcited by 11
  10. 2025Think Twice, Generate Once: Safeguarding by Progressive Self-Reflectioncited by 11
  11. 2025Refusal Falls off a Cliff: How Safety Alignment Fails in Reasoning?cited by 11
  12. 2025Test-Time Scaling in Reasoning Models Is Not Effective for Knowledge-Intensive Tasks Yetcited by 10
  13. 2025SecInfer: Preventing Prompt Injection via Inference-time Scalingcited by 10
  14. 2025AegisLLM: Scaling Agentic Systems for Self-Reflective Defense in LLM Securitycited by 9
  15. 2025Jailbreaking to Jailbreakcited by 8
  16. 2025Beyond Safe Answers: A Benchmark for Evaluating True Risk Awareness in Large Reasoning Modelscited by 7
  17. 2025RedDebate: Safer Responses Through Multi-Agent Red Teaming Debatescited by 7
  18. 2025Beyond Data Privacy: New Privacy Risks for Large Language Modelscited by 6
  19. 2025OVERT: A Benchmark for Over-Refusal Evaluation on Text-to-Image Modelscited by 5
  20. 2025Test-Time Canonicalization by Foundation Models for Robust Perceptioncited by 5
  21. 2025Reasoning Up the Instruction Ladder for Controllable Language Modelscited by 5
  22. 2025SCGAgent: Recreating the Benefits of Reasoning Models for Secure Code Generation with Agentic Workflowscited by 4
  23. 2025SafeSearch: Automated Red-Teaming of LLM-Based Search Agentscited by 4
  24. 2025Safety Must Precede the Deployment of Open-Ended AIcited by 3
  25. 2025Towards Safety Reasoning in LLMs: AI-agentic Deliberation for Policy-embedded CoT Data Creationcited by 3
  26. 2025Where Do Reasoning Models Refuse?cited by 3
  27. 2025Evaluating Language Model Reasoning about Confidential Informationcited by 3
  28. 2025Reasoning Introduces New Poisoning Attacks Yet Makes Them More Complicatedcited by 3
  29. 2025Does More Inference-Time Compute Really Help Robustness?cited by 2
  30. 2025Red Teaming Large Reasoning Modelscited by 2
  31. 2026STAR-S: Improving Safety Alignment through Self-Taught Reasoning on Safety Rulescited by 2
  32. 2026Jailbreak Scaling Laws for Large Language Models: Polynomial-Exponential Crossovercited by 2
  33. 2026Contrastive Reasoning Alignment: Reinforcement Learning from Hidden Representationscited by 2
  34. 2025A Cryptographic Perspective on Mitigation vs. Detection in Machine Learningcited by 1
  35. 2025EVALOOOP: A Self-Consistency-Centered Framework for Assessing Large Language Model Robustness in Programmingcited by 1
  36. 2025The Limits of Preference Data for Post-Trainingcited by 1
  37. 2025Red Teaming Multimodal Language Models: Evaluating Harm Across Prompt Modalities and Modelscited by 1
  38. 2025From Perception to Reasoning: Deep Thinking Empowers Multimodal Large Language Modelscited by 1

50 most cited of 60 All 60 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Adversarial Reasoning at Jailbreaking Time
  2. 2026Consistency of Large Reasoning Models Under Multi-Turn Attacks
  3. 2026Assessing Adversarial Robustness of Latent Reasoning Models
  4. 2026On the Efficiency-Safety Dilemma in Large Reasoning Models
  5. 2025AdvChain: Adversarial Chain-of-Thought Tuning for Robust Safety Alignment of Large Reasoning Models
  6. 2025Distractor Injection Attacks on Large Reasoning Models: Characterization and Defense
  7. 2025Output Length Effect on DeepSeek-R1's Safety in Forced Thinking
  8. 2025Benchmarking Gaslighting Negation Attacks Against Reasoning Models