# Claudini: Autoresearch Discovers State-of-the-Art Adversarial Attack Algorithms for LLMs

arXiv: https://arxiv.org/abs/2603.24511 · HTML: https://mlsanity.com/paper/2603.24511/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security.
- Builds on 14 of its 63 references that are in the map.
- First posted on arXiv 25 Mar 2026.

## Placed in

- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Builds on (most cited first, 14 of 14)

- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [gpt-oss-120b & gpt-oss-20b Model Card](https://mlsanity.com/paper/2508.10925/) (2025)
- [AlphaEvolve: A coding agent for scientific and algorithmic discovery](https://mlsanity.com/paper/2506.13131/) (2025)
- [KernelBench: Can LLMs Write Efficient GPU Kernels?](https://mlsanity.com/paper/2502.10517/) (2025)
- [The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injections](https://mlsanity.com/paper/2510.09023/) (2025)
- [PostTrainBench: Can LLM Agents Automate LLM Post-Training?](https://mlsanity.com/paper/2603.08640/) (2026)
- [PAL: Proxy-Guided Black-Box Attack on Large Language Models](https://mlsanity.com/paper/2402.09674/) (2024)
- [Faster-GCG: Efficient Discrete Optimization Jailbreak Attacks against Aligned Large Language Models](https://mlsanity.com/paper/2410.15362/) (2024)
- [How Vulnerable Are AI Agents to Indirect Prompt Injections? Insights from a Large-Scale Public Competition](https://mlsanity.com/paper/2603.15714/) (2026)
- [Meta SecAlign: A Secure Foundation LLM Against Prompt Injection Attacks](https://mlsanity.com/paper/2507.02735/) (2025)
- [AgenticRed: Evolving Agentic Systems for Red-Teaming](https://mlsanity.com/paper/2601.13518/) (2026)
- [The Resurgence of GCG Adversarial Attacks on Large Language Models](https://mlsanity.com/paper/2509.00391/) (2025)
- [Jailbreaking LLMs Without Gradients or Priors: Effective and Transferable Attacks](https://mlsanity.com/paper/2601.03420/) (2026)

## Built on by (most cited first, 17 of 17)

- [MLS-Bench: A Holistic and Rigorous Assessment of AI Systems on Building Better AI](https://mlsanity.com/paper/2605.08678/) (2026)
- [Omni-SimpleMem: Autoresearch-Guided Discovery of Lifelong Multimodal Agent Memory](https://mlsanity.com/paper/2604.01007/) (2026)
- [Stealing Reasoning Traces from Proprietary LLM APIs](https://mlsanity.com/paper/2608.09867/) (2026)
- [WMAttack: Automated Attack Search for Adversarial Evaluation of World-Model Agents](https://mlsanity.com/paper/2605.23220/) (2026)
- [SwarmResearch: Orchestrating Coding Agents for Open-Ended Discovery](https://mlsanity.com/paper/2607.02807/) (2026)
- [Memory-Induced Tool-Drift in LLM Agents](https://mlsanity.com/paper/2605.24941/) (2026)
- [Agent Hacks Agent: Autoresearch for Production-Agent Red-Teaming](https://mlsanity.com/paper/2607.11698/) (2026)
- [ResearchArena: Evaluating Sabotage and Monitoring in Automated AI R&D](https://mlsanity.com/paper/2607.19321/) (2026)
- [Workflow Closure Is Not Scientific Closure in Auto-Research Systems](https://mlsanity.com/paper/2605.26200/) (2026)
- [FAPO: Fully Automated Prompt Optimization of Multi-Step LLM Pipelines](https://mlsanity.com/paper/2606.19605/) (2026)
- [RecEvolve: A Knowledge-Driven Autonomous Agent System for Recommender Systems](https://mlsanity.com/paper/2609.01622/) (2026)
- [Applying an Agentic Coding Tool for Improving Published Algorithm Implementations](https://mlsanity.com/paper/2604.13109/) (2026)
- [The Evaluation Game: Beyond Static LLM Benchmarking](https://mlsanity.com/paper/2605.19377/) (2026)
- [Alignment via Training Against Probes Without Losing Monitorability](https://mlsanity.com/paper/2609.38645/) (2026)
- [Exploring Jailbreak Vulnerability Across Depth-Based Dynamic Inference in Large Language Models](https://mlsanity.com/paper/2610.15207/) (2026)
- [EAU: Environment-Agent-User Co-evolution for Training User-Centric Agents](https://mlsanity.com/paper/2610.16523/) (2026)
- [Large Language Models Extend Known Research But Do Not Create New Research Directions](https://mlsanity.com/paper/2610.17439/) (2026)

## Abstract (arXiv)

We show that AI agents are capable of discovering novel algorithms for adversarial attacks against LLMs, advancing the state of the art on white-box jailbreaking and prompt injection evaluations. We deploy frontier agents, such as Claude Code and Codex, in an autoresearch loop with access to a library of 30+ prior methods and an evaluation script with a fixed compute budget. We show this pipeline to be effective in jailbreaking OpenAI's GPT-OSS-Safeguard-20B and in prompt injections against Meta-SecAlign-70B, an adversarially robust model. For GPT-OSS-Safeguard, the best agent-discovered method achieves up to 80% attack success rate on CBRN queries, compared to <50% for existing methods. For SecAlign, it achieves 100% ASR, while the best prior automated methods only achieve 82%. Notably, in our setting, attack methods are developed on unrelated surrogate models for a pure random-target token-forcing task, yet generalize directly to prompt injection on the adversarially trained model. Finally, we trace the lineage of methods developed during autoresearch, characterizing the agents' strategies and failure modes. Adversarial ML has long held that defenses must be evaluated against attacks tailored to them; autoresearch automates this principle, and we argue it should be the minimum bar for defense evaluation going forward.