TroubleLLM: Align to Red Team Expert
On the map: LLM safety and security → Safety alignment, refusal and guardrails
Abstract (arXiv)
Large Language Models (LLMs) become the start-of-the-art solutions for a variety of natural language tasks and are integrated into real-world applications. However, LLMs can be potentially harmful in manifesting undesirable safety issues like social biases and toxic content. It is imperative to assess its safety issues before deployment. However, the quality and diversity of test prompts generated by existing methods are still far from satisfactory. Not only are these methods labor-intensive and require large budget costs, but the controllability of test prompt generation is lacking for the specific testing domain of LLM applications. With the idea of LLM for LLM testing, we propose the first LLM, called TroubleLLM, to generate controllable test prompts on LLM safety issues. Extensive experiments and human evaluation illustrate the superiority of TroubleLLM on generation quality and generation controllability.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 19 of its 28 references that are in the map. First posted on arXiv 28 Feb 2024.
Builds on
19 of 28 references.
References · 19
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
- 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
- 2022Large Language Models Encode Clinical Knowledgecited by 1,807
- 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
- 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
7 more
- 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 591
- 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
- 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 484
- 2023RRHF: Rank Responses to Align Language Models with Human Feedback without tearscited by 454
- 2021Towards Understanding and Mitigating Social Biases in Language Modelscited by 294
- 2019Measuring Bias in Contextualized Word Representationscited by 275
- 2022Is Neural Topic Modelling Better than Clustering? An Empirical Study on Clustering with Contextual Embeddings for Topicscited by 35
Built on by
Citing papers per year: 2024: 1; 2025: 1.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025ASTRAL: Automated Safety Testing of Large Language Models
- 2025LLM-Safety Evaluations Lack Robustness
- 2024Testing and Evaluation of Large Language Models: Correctness, Non-Toxicity, and Fairness
- 2024Large Language Model Safety: A Holistic Survey
- 2024SG-Bench: Evaluating LLM Safety Generalization Across Diverse Tasks and Prompt Types
- 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment
- 2024Agent-SafetyBench: Evaluating the Safety of LLM Agents
- 2023Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignment