mlsanityData to 7 Oct 2026

No Offense Taken: Eliciting Offensiveness from Language Models

arXiv 2310.00892 ↗First posted 2 Oct 2023cs.CL cs.AIcited by 3

On the map: LLM safety and security → Prompt injection and agent security

Abstract (arXiv)

This work was completed in May 2022. For safe and reliable deployment of language models in the real world, testing needs to be robust. This robustness can be characterized by the difficulty and diversity of the test cases we evaluate these models on. Limitations in human-in-the-loop test case generation has prompted an advent of automated test case generation approaches. In particular, we focus on Red Teaming Language Models with Language Models by Perez et al.(2022). Our contributions include developing a pipeline for automated test case generation via red teaming that leverages publicly available smaller language models (LMs), experimenting with different target LMs and red classifiers, and generating a corpus of test cases that can help in eliciting offensive responses from widely deployed LMs and identifying their failure modes.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 25 of its 32 references that are in the map. First posted on arXiv 2 Oct 2023.

Builds on

25 of 32 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Red Teaming Language Models with Language Models7 Feb

Other references · 24

  1. 2017Attention Is All You Needcited by 59,932
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  4. 2020Learning to summarize from human feedbackcited by 2,562
  5. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  6. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  7. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
  8. 2021Ethical and social risks of harm from Language Modelscited by 957
  9. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  10. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  11. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
  12. 2020Recipes for building an open-domain chatbotcited by 593
12 more
  1. 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 484
  2. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 410
  3. 2021Towards Understanding and Mitigating Social Biases in Language Modelscited by 294
  4. 2021Internet-Augmented Dialogue Generationcited by 217
  5. 2020HateCheck: Functional Tests for Hate Speech Detection Modelscited by 215
  6. 2019Build it Break it Fix it for Dialogue Safety: Robustness from Adversarial Human Attackcited by 176
  7. 2020Recipes for Safety in Open-domain Chatbotscited by 168
  8. 2017ParlAI: A Dialog Research Software Platformcited by 139
  9. 2020Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehensioncited by 131
  10. 2021Improving Question Answering Model Robustness with Synthetic Adversarial Data Generationcited by 77
  11. 2019The Dialogue Dodecathlon: Open-Domain Knowledge and Image Grounded Conversational Agentscited by 45
  12. 2021Analyzing Dynamic Adversarial Training Data in the Limitcited by 24

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 1.

3 papers.

  1. 2024Red-Teaming for Generative AI: Silver Bullet or Security Theater?cited by 81
  2. 2024Recent advancements in LLM Red-Teaming: Techniques, Defenses, and Ethical Considerationscited by 8
  3. 2025ToxSearch: Evolving Prompts for Toxicity Search in Large Language Modelscited by 0