mlsanityData to 7 Oct 2026

On the Challenges of Using Black-Box APIs for Toxicity Evaluation in Research

arXiv 2304.12397 ↗First posted 24 Apr 2023cs.CL cs.AIcited by 44

Not on the map. Why?

Abstract (arXiv)

Perception of toxicity evolves over time and often differs between geographies and cultural backgrounds. Similarly, black-box commercially available APIs for detecting toxicity, such as the Perspective API, are not static, but frequently retrained to address any unattended weaknesses and biases. We evaluate the implications of these changes on the reproducibility of findings that compare the relative merits of models and methods that aim to curb toxicity. Our findings suggest that research that relied on inherited automatic toxicity scores to compare models and techniques may have resulted in inaccurate findings. Rescoring all models from HELM, a widely respected living benchmark, for toxicity with the recent version of the API led to a different ranking of widely used foundation models. We suggest caution in applying apples-to-apples comparisons between studies and lay recommendations for a more structured approach to evaluating toxicity over time. Code and data are available at https://github.com/for-ai/black-box-api-challenges.

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). Builds on 21 of its 41 references that are in the map. First posted on arXiv 24 Apr 2023.

Builds on

21 of 41 references.

References · 21

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  3. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  4. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  5. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  6. 2018Model Cards for Model Reportingcited by 1,068
  7. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
  8. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
  9. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 557
  10. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 434
  11. 2020GeDi: Generative Discriminator Guided Sequence Generationcited by 377
  12. 2019Racial Bias in Hate Speech and Abusive Language Detection Datasetscited by 208
9 more
  1. 2022A New Generation of Perspective API: Efficient Multilingual Character-level Transformerscited by 181
  2. 2021Challenges in Detoxifying Language Modelscited by 169
  3. 2022Exploring the Limits of Domain-Adaptive Training for Detoxifying Large-Scale Language Modelscited by 67
  4. 2022Is Your Toxicity My Toxicity? Exploring the Impact of Rater Identity on Toxicity Annotationcited by 67
  5. 2017Like trainer, like bot? Inheritance of bias in algorithmic content moderationcited by 34
  6. 2022Unified Detoxifying and Debiasing in Language Generation via Inference-time Adaptive Optimizationcited by 32
  7. 2022Metadata Archaeology: Unearthing Data Subsets by Leveraging Training Dynamicscited by 26
  8. 2022Reward Modeling for Mitigating Toxicity in Transformer-based Language Modelscited by 17
  9. 2018But Who Protects the Moderators? The Case of Crowdsourced Image Moderationcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 15; 2024: 21; 2025: 5; 2026: 3.

44 papers.

Sort:

By area:

  1. 2023PaLM 2 Technical Reportcited by 1,154
  2. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  3. 2023Prometheus: Inducing Fine-grained Evaluation Capability in Language Modelscited by 558
  4. 2023A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicitycited by 180
  5. 2023OctoPack: Instruction Tuning Code Large Language Modelscited by 178
  6. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 123
  7. 2023Evaluating the Social Impact of Generative AI Systems in Systems and Societycited by 91
  8. 2023Reward-Augmented Decoding: Efficient Controlled Text Generation With a Unidirectional Reward Modelcited by 79
  9. 2023The Data Provenance Initiative: A Large Scale Audit of Dataset Licensing & Attribution in AIcited by 73
  10. 2023Locally Differentially Private Document Generation Using Zero Shot Promptingcited by 63
  11. 2023Elo Uncovered: Robustness and Best Practices in Language Model Evaluationcited by 58
  12. 2024Consent in Crisis: The Rapid Decline of the AI Data Commonscited by 45
32 more
  1. 2023Open, Closed, or Small Language Models for Text Classification?cited by 34
  2. 2024Uncovering Bias in Large Vision-Language Models at Scale with Counterfactualscited by 27
  3. 2023Goodtriever: Adaptive Toxicity Mitigation with Retrieval-augmented Modelscited by 23
  4. 2024Can LLMs Recognize Toxicity? A Structured Investigation Framework and Toxicity Metriccited by 23
  5. 2023Which Prompts Make The Difference? Data Prioritization For Efficient Human LLM Evaluationcited by 18
  6. 2024From One to Many: Expanding the Scope of Toxicity Mitigation in Language Modelscited by 18
  7. 2023Bring Your Own Data! Self-Supervised Evaluation for Large Language Modelscited by 17
  8. 2024Walking in Others' Shoes: How Perspective-Taking Guides Large Language Models in Reducing Toxicity and Biascited by 17
  9. 2024PolygloToxicityPrompts: Multilingual Evaluation of Neural Toxic Degeneration in Large Language Modelscited by 16
  10. 2024Diffusion Guided Language Modelingcited by 15
  11. 2024HateDay: Insights from a Global Hate Speech Dataset Representative of a Day on Twittercited by 12
  12. 2024RSA-Control: A Pragmatics-Grounded Lightweight Controllable Text Generation Frameworkcited by 11
  13. 2024DESTEIN: Navigating Detoxification of Language Models via Universal Steering Pairs and Head-wise Activation Fusioncited by 9
  14. 2024Large Language Models can be Strong Self-Detoxifierscited by 7
  15. 2024Toxicity of the Commons: Curating Open-Source Pre-Training Datacited by 7
  16. 2024LLM See, LLM Do: Guiding Data Generation to Target Non-Differentiable Objectivescited by 6
  17. 2024FrenchToxicityPrompts: a Large Benchmark for Evaluating and Mitigating Toxicity in French Textscited by 5
  18. 2024Cross-Lingual Transfer of Debiasing and Detoxification in Multilingual LLMs: An Extensive Investigationcited by 5
  19. 2025Aligned Probing: Relating Toxic Behavior and Model Internalscited by 5
  20. 2025Evaluating and Improving Robustness in Large Language Models: A Survey and Future Directionscited by 5
  21. 2024Investigating Adversarial Trigger Transfer in Large Language Modelscited by 4
  22. 2024Exploring Human-LLM Conversations: Mental Models and the Originator of Toxicitycited by 4
  23. 2024Hatred Stems from Ignorance! Distillation of the Persuasion Modes in Countering Conversational Hate Speechcited by 2
  24. 2024Leveraging Open-Source Large Language Models for Native Language Identificationcited by 2
  25. 2025Designing Synthetic Discussion Generation Systems: A Case Study for Online Facilitationcited by 2
  26. 2025Something Just Like TRuST : Toxicity Recognition of Span and Targetcited by 2
  27. 2026Test-Time Detoxification without Training or Learning Anythingcited by 2
  28. 2024Characterization of Political Polarized Users Attacked by Language Toxicity on Twittercited by 1
  29. 2026Bye Bye Perspective API: Lessons for Building and Governing Measurement Infrastructurecited by 1
  30. 2023Breaking the Silence: the Threats of Using LLMs in Software Engineeringcited by 0
  31. 2025Hatevolution: What Static Benchmarks Don't Tell Uscited by 0
  32. 2026Beyond Static Benchmarks: Synthesizing Harmful Content via Persona-based Simulation for Robust Evaluationcited by 0