# The Human Factor in AI Red Teaming: Perspectives from Social and Collaborative Computing

arXiv: https://arxiv.org/abs/2407.07786 · HTML: https://mlsanity.com/paper/2407.07786/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security.
- Builds on 3 of its 25 references that are in the map.
- First posted on arXiv 10 Jul 2024.

## Placed in

- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Builds on (most cited first, 3 of 3)

- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [Studying Up Machine Learning Data: Why Talk About Bias When We Mean Power?](https://mlsanity.com/paper/2109.08131/) (2021)
- [Responsible AI Research Needs Impact Statements Too](https://mlsanity.com/paper/2311.11776/) (2023)

## Built on by (most cited first, 8 of 8)

- [Perceptions of Sentient AI and Other Digital Minds: Evidence from the AI, Morality, and Sentience (AIMS) Survey](https://mlsanity.com/paper/2407.08867/) (2024)
- [Exploring Empty Spaces: Human-in-the-Loop Data Augmentation](https://mlsanity.com/paper/2410.01088/) (2024)
- [Provocation: Who benefits from "inclusion" in Generative AI?](https://mlsanity.com/paper/2411.09102/) (2024)
- [From Firewalls to Frontiers: AI Red-Teaming is a Domain-Specific Evolution of Cyber Red-Teaming](https://mlsanity.com/paper/2509.11398/) (2025)
- [AI red-teaming is a sociotechnical problem: on values, labor, and harms](https://mlsanity.com/paper/2412.09751/) (2024)
- [A Conceptual Framework for AI Capability Evaluations](https://mlsanity.com/paper/2506.18213/) (2025)
- [Red Teaming LLMs as Socio-Technical Practice: From Exploration and Data Creation to Evaluation](https://mlsanity.com/paper/2602.18483/) (2026)
- [Position: Preventing AI-Generated CSAM Necessitates New Approaches to AI Safety](https://mlsanity.com/paper/2607.05407/) (2026)

## Abstract (arXiv)

Rapid progress in general-purpose AI has sparked significant interest in "red teaming," a practice of adversarial testing originating in military and cybersecurity applications. AI red teaming raises many questions about the human factor, such as how red teamers are selected, biases and blindspots in how tests are conducted, and harmful content's psychological effects on red teamers. A growing body of HCI and CSCW literature examines related practices-including data labeling, content moderation, and algorithmic auditing. However, few, if any have investigated red teaming itself. Future studies may explore topics ranging from fairness to mental health and other areas of potential harm. We aim to facilitate a community of researchers and practitioners who can begin to meet these challenges with creativity, innovation, and thoughtful reflection.