# Learning What to Fail On: Failure-Mode Contextual Bandits for Adversarial Data Curation

arXiv: https://arxiv.org/abs/2608.18681 · HTML: https://mlsanity.com/paper/2608.18681/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 24 of its 35 references that are in the map.
- First posted on arXiv 19 Aug 2026.

## Placed in

- Retrieval-augmented generation / [Privacy and security of RAG](https://mlsanity.com/rag/privacy-and-security-of-rag/)

## Builds on (most cited first, 24 of 24)

- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation](https://mlsanity.com/paper/2402.03216/) (2024)
- [Adversarial NLI: A New Benchmark for Natural Language Understanding](https://mlsanity.com/paper/1910.14599/) (2019)
- [SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model](https://mlsanity.com/paper/2502.02737/) (2025)
- [Adversarial Example Generation with Syntactically Controlled Paraphrase Networks](https://mlsanity.com/paper/1804.06059/) (2018)
- [Breaking NLI Systems with Sentences that Require Simple Lexical Inferences](https://mlsanity.com/paper/1805.02266/) (2018)
- [Fine-grained Fact Verification with Kernel Graph Attention Network](https://mlsanity.com/paper/1910.09796/) (2019)
- [GEAR: Graph-based Evidence Aggregating and Reasoning for Fact Verification](https://mlsanity.com/paper/1908.01843/) (2019)
- [Adversarially Regularising Neural NLI Models to Integrate Logical Background Knowledge](https://mlsanity.com/paper/1808.08609/) (2018)
- [Prompting Large Language Models for Counterfactual Generation: An Empirical Study](https://mlsanity.com/paper/2305.14791/) (2023)
- [Learning What Data to Learn](https://mlsanity.com/paper/1702.08635/) (2017)
- [Reinforced Curriculum Learning on Pre-trained Neural Machine Translation Models](https://mlsanity.com/paper/2004.05757/) (2020)
- [Behavior Analysis of NLI Models: Uncovering the Influence of Three Factors on Robustness](https://mlsanity.com/paper/1805.04212/) (2018)
- [A synthetic data approach for domain generalization of NLI models](https://mlsanity.com/paper/2402.12368/) (2024)
- [BEVERS: A General, Simple, and Performant Framework for Automatic Fact Verification](https://mlsanity.com/paper/2303.16974/) (2023)
- [VAULT: Vigilant Adversarial Updates via LLM-Driven Retrieval-Augmented Generation for NLI](https://mlsanity.com/paper/2508.00965/) (2025)
- [Don't Lag, RAG: Training-Free Adversarial Detection Using RAG](https://mlsanity.com/paper/2504.04858/) (2025)
- [RL-Selector: Reinforcement Learning-Guided Data Selection via Redundancy Assessment](https://mlsanity.com/paper/2506.21037/) (2025)
- [Extracting and filtering paraphrases by bridging natural language inference and paraphrasing](https://mlsanity.com/paper/2111.07119/) (2021)
- [Enhancing Object Detection Robustness: Detecting and Restoring Confidence in the Presence of Adversarial Patch Attacks](https://mlsanity.com/paper/2403.12988/) (2024)
- [Seeing Isn't Believing: Context-Aware Adversarial Patch Synthesis via Conditional GAN](https://mlsanity.com/paper/2509.22836/) (2025)
- [RL-Guided Data Selection for Language Model Finetuning](https://mlsanity.com/paper/2509.25850/) (2025)

## Abstract (arXiv)

We introduce a failure-aware adversarial retrieval-augmented framework for improving robustness in natural language understanding. Rather than selecting synthetic examples with a fixed reward threshold, our method formulates adversarial data curation as a failure-mode contextual bandit problem. Candidate examples are generated with retrieval-augmented prompting, filtered by the current target model, automatically validated by an LLM judge ensemble, and clustered into recurring failure modes. A stochastic policy then selects which failure modes to sample for retraining, and is updated using validation-based reward that balances robustness gains, forgetting, and data cost. This makes the data curator itself the learning agent, enabling adaptive selection of the most useful model failures across training rounds. On standard benchmarks, our approach improves RoBERTa-base accuracy from 88.48% to 92.60% on SNLI, from 75.04% to 80.95% on ANLI, and from 54.67% to 71.99% on MultiNLI, while consistently outperforming prior adversarial augmentation methods. We further demonstrate transfer to FEVER fact verification, achieving up to 79.86% FEVER score and 82.45% accuracy with RoBERTa-large. Finally, we provide a theoretical interpretation showing that, under stated assumptions, failure-mode sampling can reduce shortcut-aligned gradient contributions while inducing bounded distributional drift. By combining retrieval, automated validation, contextual-bandit failure selection, and controlled adversarial retraining, our framework enables scalable robustness improvement without additional human annotation.