# AI red-teaming is a sociotechnical problem: on values, labor, and harms

arXiv: https://arxiv.org/abs/2412.09751 · HTML: https://mlsanity.com/paper/2412.09751/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 40 references that are in the map.
- First posted on arXiv 12 Dec 2024.

## Builds on (most cited first, 8 of 8)

- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Sociotechnical Safety Evaluation of Generative AI Systems](https://mlsanity.com/paper/2310.11986/) (2023)
- [Red-Teaming for Generative AI: Silver Bullet or Security Theater?](https://mlsanity.com/paper/2401.15897/) (2024)
- [Participation in the age of foundation models](https://mlsanity.com/paper/2405.19479/) (2024)
- [Safety and Fairness for Content Moderation in Generative Models](https://mlsanity.com/paper/2306.06135/) (2023)
- [The Human Factor in AI Red Teaming: Perspectives from Social and Collaborative Computing](https://mlsanity.com/paper/2407.07786/) (2024)
- [AURA: Amplifying Understanding, Resilience, and Awareness for Responsible AI Content Work](https://mlsanity.com/paper/2411.01426/) (2024)
- [Provocation: Who benefits from "inclusion" in Generative AI?](https://mlsanity.com/paper/2411.09102/) (2024)

## Abstract (arXiv)

As generative AI technologies find more and more real-world applications, the importance of testing their performance and safety seems paramount. "Red-teaming" has quickly become the primary approach to test AI models--prioritized by AI companies, and enshrined in AI policy and regulation. Members of red teams act as adversaries, probing AI systems to test their safety mechanisms and uncover vulnerabilities. Yet we know far too little about this work or its implications. This essay calls for collaboration between computer scientists and social scientists to study the sociotechnical systems surrounding AI technologies, including the work of red-teaming, to avoid repeating the mistakes of the recent past. We highlight the importance of understanding the values and assumptions behind red-teaming, the labor arrangements involved, and the psychological impacts on red-teamers, drawing insights from the lessons learned around the work of content moderation.