mlsanityData to 14 Oct 2026

Down the Toxicity Rabbit Hole: A Novel Framework to Bias Audit Large Language Models

arXiv 2309.06415 ↗First posted 8 Sep 2023cs.CL cs.CYcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper makes three contributions. First, it presents a generalizable, novel framework dubbed textittoxicity rabbit hole that iteratively elicits toxic content from a wide suite of large language models. Spanning a set of 1,266 identity groups, we first conduct a bias audit of textttPaLM 2 guardrails presenting key insights. Next, we report generalizability across several other models. Through the elicited toxic content, we present a broad analysis with a key emphasis on racism, antisemitism, misogyny, Islamophobia, homophobia, and transphobia. Finally, driven by concrete examples, we discuss potential ramifications.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 49 references that are in the map. First posted on arXiv 8 Sep 2023.

Builds on

21 of 49 references.

References · 21

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  5. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  6. 2016Enriching Word Vectors with Subword Informationcited by 1,583
  7. 2023PaLM 2 Technical Reportcited by 1,162
  8. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,125
  9. 2022Improving alignment of dialogue agents via targeted human judgementscited by 531
  10. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 498
  11. 2019Social Bias Frames: Reasoning about Social and Power Implications of Languagecited by 380
  12. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 377
9 more
  1. 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 360
  2. 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 333
  3. 2023How is ChatGPT's behavior changing over time?cited by 270
  4. 2023Pretraining Language Models with Human Preferencescited by 247
  5. 2023Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMscited by 184
  6. 2022Picking on the Same Person: Does Algorithmic Monoculture lead to Outcome Homogenization?cited by 91
  7. 2019Analyzing Polarization in Social Media: Method and Application to Tweets on 21 Mass Shootingscited by 67
  8. 2020A Framework for the Computational Linguistic Analysis of Dehumanizationcited by 31
  9. 2020We Don't Speak the Same Language: Interpreting Polarization through Machine Translationcited by 23

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Something Just Like TRuST : Toxicity Recognition of Span and Target
  2. 2025Towards Safer Pretraining: Analyzing and Filtering Harmful Content in Webscale datasets for Responsible LLMs
  3. 2022Reward Modeling for Mitigating Toxicity in Transformer-based Language Models
  4. 2024Realistic Evaluation of Toxicity in Large Language Models
  5. 2026A Survey of Toxicity Detection and Mitigation Strategies for Multilingual Language Models
  6. 2026Measuring and Mitigating Toxicity in Large Language Models: A Comprehensive Replication Study
  7. 2024RTP-LX: Can LLMs Evaluate Toxicity in Multilingual Scenarios?
  8. 2026ToxiREX: A Dataset on Toxic REasoning in ConteXt