# Adaptive Instruction Composition for Automated LLM Red-Teaming

arXiv: https://arxiv.org/abs/2604.21159 · HTML: https://mlsanity.com/paper/2604.21159/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security.
- Builds on 28 of its 56 references that are in the map.
- First posted on arXiv 22 Apr 2026.

## Placed in

- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Builds on (most cited first, 28 of 28)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023)
- [HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal](https://mlsanity.com/paper/2402.04249/) (2024)
- [Jailbreaking Black Box Large Language Models in Twenty Queries](https://mlsanity.com/paper/2310.08419/) (2023)
- [Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations](https://mlsanity.com/paper/2312.06674/) (2023)
- [Universal Adversarial Triggers for Attacking and Analyzing NLP](https://mlsanity.com/paper/1908.07125/) (2019)
- [Pretraining Language Models with Human Preferences](https://mlsanity.com/paper/2302.08582/) (2023)
- [MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbots](https://mlsanity.com/paper/2307.08715/) (2023)
- [MART: Improving LLM Safety with Multi-round Automatic Red-Teaming](https://mlsanity.com/paper/2311.07689/) (2023)
- [Evaluating the Evaluation of Diversity in Natural Language Generation](https://mlsanity.com/paper/2004.02990/) (2020)
- [Neural Thompson Sampling](https://mlsanity.com/paper/2010.00827/) (2020)
- [A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Models](https://mlsanity.com/paper/2402.13457/) (2024)
- [Red-Teaming for Generative AI: Silver Bullet or Security Theater?](https://mlsanity.com/paper/2401.15897/) (2024)
- [Against The Achilles' Heel: A Survey on Red Teaming for Generative Models](https://mlsanity.com/paper/2404.00629/) (2024)
- [Satisficing in Time-Sensitive Bandit Learning](https://mlsanity.com/paper/1803.02855/) (2018)
- [PyRIT: A Framework for Security Risk Identification and Red Teaming in Generative AI System](https://mlsanity.com/paper/2410.02828/) (2024)
- [The Unreasonable Effectiveness of Greedy Algorithms in Multi-Armed Bandit with Many Arms](https://mlsanity.com/paper/2002.10121/) (2020)
- [Diverse and Effective Red Teaming with Auto-generated Rewards and Multi-step Reinforcement Learning](https://mlsanity.com/paper/2412.18693/) (2024)
- [A Trembling House of Cards? Mapping Adversarial Attacks against Language Agents](https://mlsanity.com/paper/2402.10196/) (2024)
- [Improving Diversity of Commonsense Generation by Large Language Models via In-Context Learning](https://mlsanity.com/paper/2404.16807/) (2024)
- [Attack Atlas: A Practitioner's Perspective on Challenges and Pitfalls in Red Teaming GenAI](https://mlsanity.com/paper/2409.15398/) (2024)
- [Exploring Straightforward Conversational Red-Teaming](https://mlsanity.com/paper/2409.04822/) (2024)

## Built on by (most cited first, 3 of 3)

- [Workspace Topology as an Attack Vector in Agentic Coding Assistants](https://mlsanity.com/paper/2608.14876/) (2026)
- [Structural Jailbreaks Generalize but Do Not Compound: A cross-provider and multilingual study of Involuntary In-Context Learning](https://mlsanity.com/paper/2609.08373/) (2026)
- [FinRT: Distilling Adaptive Red-Teaming Strategies into Reusable Adversarial Generators in Consumer Finance](https://mlsanity.com/paper/2609.36474/) (2026)

## Abstract (arXiv)

Many approaches to LLM red-teaming leverage an attacker LLM to discover jailbreaks against a target. Several of them task the attacker with identifying effective strategies through trial and error, resulting in a semantically limited range of successes. Another approach discovers diverse attacks by combining crowdsourced harmful queries and tactics into instructions for the attacker, but does so at random, limiting effectiveness. This article introduces a novel framework, Adaptive Instruction Composition, that combines crowdsourced texts according to an adaptive mechanism trained to jointly optimize effectiveness with diversity. We use reinforcement learning to balance exploration with exploitation in a combinatorial space of instructions to guide the attacker toward diverse generations tailored to target vulnerabilities. We demonstrate that our approach substantially outperforms random combination on a set of effectiveness and diversity metrics, even under model transfer. Further, we show that it surpasses a host of recent adaptive approaches on Harmbench. We employ a lightweight neural contextual bandit that adapts to contrastive embedding inputs, and provide ablations suggesting that the contrastive pretraining enables the network to rapidly generalize and scale to the massive space as it learns.