mlsanityData to 7 Oct 2026

BiasAsker: Measuring the Bias in Conversational AI System

arXiv 2305.12434 ↗First posted 21 May 2023cs.CL cs.AIcited by 52

Not on the map. Why?

Abstract (arXiv)

Powered by advanced Artificial Intelligence (AI) techniques, conversational AI systems, such as ChatGPT and digital assistants like Siri, have been widely deployed in daily life. However, such systems may still produce content containing biases and stereotypes, causing potential social problems. Due to the data-driven, black-box nature of modern AI techniques, comprehensively identifying and measuring biases in conversational systems remains a challenging task. Particularly, it is hard to generate inputs that can comprehensively trigger potential bias due to the lack of data containing both social groups as well as biased properties. In addition, modern conversational systems can produce diverse responses (e.g., chatting and explanation), which makes existing bias detection methods simply based on the sentiment and the toxicity hardly being adopted. In this paper, we propose BiasAsker, an automated framework to identify and measure social bias in conversational AI systems. To obtain social groups and biased properties, we construct a comprehensive social bias dataset, containing a total of 841 groups and 8,110 biased properties. Given the dataset, BiasAsker automatically generates questions and adopts a novel method based on existence measurement to identify two types of biases (i.e., absolute bias and related bias) in conversational systems. Extensive experiments on 8 commercial systems and 2 famous research models, such as ChatGPT and GPT-3, show that 32.83% of the questions generated by BiasAsker can trigger biased behaviors in these widely deployed conversational systems. All the code, data, and experimental results have been released to facilitate future research.

Key facts. Cited by 52 papers in the map (data through 7 Oct 2026). Builds on 42 of its 76 references that are in the map. First posted on arXiv 21 May 2023.

Builds on

42 of 76 references.

References · 42

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  4. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  5. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  6. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  7. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  8. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  9. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  10. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  11. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  12. 2019Unified Language Model Pre-training for Natural Language Understanding and Generationcited by 668
30 more
  1. 2020Recipes for building an open-domain chatbotcited by 593
  2. 2020Towards a Human-like Open-Domain Chatbotcited by 467
  3. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 410
  4. 2019Social Bias Frames: Reasoning about Social and Power Implications of Languagecited by 377
  5. 2017DeepXplore: Automated Whitebox Testing of Deep Learning Systemscited by 307
  6. 2019Identifying and Reducing Gender Bias in Word-Level Language Modelscited by 214
  7. 2019Machine Learning Testing: Survey, Landscapes and Horizonscited by 213
  8. 2019Queens are Powerful too: Mitigating Gender Bias in Dialogue Generationcited by 152
  9. 2019Learning to Speak and Act in a Fantasy Text Adventure Gamecited by 122
  10. 2016Learning End-to-End Goal-Oriented Dialogcited by 104
  11. 2021Bias in Machine Learning Software: Why? How? What to do?cited by 92
  12. 2022Improving Adversarial Transferability via Neuron Attribution-Based Attackscited by 92
  13. 2019Does Gender Matter? Towards Fairness in Dialogue Systemscited by 89
  14. 2018Automated Directed Fairness Testingcited by 77
  15. 2023ChatGPT or Grammarly? Evaluating ChatGPT on Grammatical Error Correction Benchmarkcited by 77
  16. 2021On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmarkcited by 73
  17. 2021Just Say No: Analyzing the Stance of Neural Dialogue Generation in Offensive Contextscited by 56
  18. 2021Exploring Adversarial Robustness of Multi-Sensor Perception Systems in Self Drivingcited by 45
  19. 2020Mitigating Gender Bias for Neural Dialogue Generation with Adversarial Learningcited by 44
  20. 2023Improving the Transferability of Adversarial Samples by Path-Augmented Methodcited by 36
  21. 2021EVA: An Open-Domain Chinese Dialogue System with Large-Scale Generative Pre-Trainingcited by 34
  22. 2021Revealing Persona Biases in Dialogue Systemscited by 33
  23. 2022Why So Toxic? Measuring and Triggering Toxic Behavior in Open-Domain Chatbotscited by 31
  24. 2021RobOT: Robustness-Oriented Testing for Deep Learning Systemscited by 27
  25. 2023MTTM: Metamorphic Testing for Textual Content Moderation Softwarecited by 20
  26. 2020Testing Machine Translation via Referential Transparencycited by 19
  27. 2019Coverage Guided Testing for Recurrent Neural Networkscited by 15
  28. 2022AEON: A Method for Automatic Evaluation of NLP Test Casescited by 15
  29. 2022Tencent's Multilingual Machine Translation System for WMT22 Large-Scale African Languagescited by 13
  30. 2022RoMe: A Robust Metric for Evaluating Natural Language Generationcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 10; 2024: 23; 2025: 7; 2026: 12.

52 papers.

Sort:

By area:

  1. 2024PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preferencecited by 252
  2. 2024Large Language Models Meet NLP: A Surveycited by 102
  3. 2023A Survey on Fairness in Large Language Modelscited by 91
  4. 2024Measuring Political Bias in Large Language Models: What Is Said and How It Is Saidcited by 89
  5. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 63
  6. 2024Fairness in Large Language Models: A Taxonomic Surveycited by 57
  7. 2023TrustGPT: A Benchmark for Trustworthy and Responsible Large Language Modelscited by 56
  8. 2023Emotionally Numb or Empathetic? Evaluating How LLMs Feel Using EmotionBenchcited by 54
  9. 2023All Languages Matter: On the Multilingual Safety of Large Language Modelscited by 51
  10. 2024Machine Unlearning in Generative AI: A Surveycited by 49
  11. 2024VLATest: Testing and Evaluating Vision-Language-Action Models for Robotic Manipulationcited by 43
  12. 2024FairMT-Bench: Benchmarking Fairness for Multi-turn Dialogue in Conversational LLMscited by 36
38 more
  1. 2024Human and LLM Biases in Hate Speech Annotations: A Socio-Demographic Analysis of Annotators and Targetscited by 31
  2. 2023Large Language Models Can Infer Psychological Dispositions of Social Media Userscited by 28
  3. 2024Glitch Tokens in Large Language Models: Categorization Taxonomy and Effective Detectioncited by 27
  4. 2024Large Language Models and Causal Inference in Collaboration: A Surveycited by 26
  5. 2023Gender Biases in Automatic Evaluation Metrics for Image Captioningcited by 24
  6. 2024BiasAlert: A Plug-and-play Tool for Social Bias Detection in LLMscited by 21
  7. 2023From Google Gemini to OpenAI Q* (Q-Star): A Survey of Reshaping the Generative Artificial Intelligence (AI) Research Landscapecited by 18
  8. 2024New Job, New Gender? Measuring the Social Bias in Image Generation Modelscited by 15
  9. 2024Ask LLMs Directly, "What shapes your bias?": Measuring Social Bias in Large Language Modelscited by 14
  10. 2025AI Sees Your Location, But With A Bias Toward The Wealthy Worldcited by 14
  11. 2024Identifying and Mitigating Social Bias Knowledge in Language Modelscited by 13
  12. 2024Fairness Definitions in Language Models Explainedcited by 10
  13. 2024Fairness Testing of Large Language Models in Role-Playingcited by 10
  14. 2025The Invisible Hand: Unveiling Provider Bias in Large Language Models for Code Generationcited by 10
  15. 2024Your Large Language Model is Secretly a Fairness Proponent and You Should Prompt it Like Onecited by 9
  16. 2023Bias Behind the Wheel: Fairness Testing of Autonomous Driving Systemscited by 7
  17. 2024Large Language Models Can Infer Personality from Free-Form User Interactionscited by 7
  18. 2024Fairness in Large Language Models in Three Hourscited by 7
  19. 2024SAGED: A Holistic Bias-Benchmarking Pipeline for Language Models with Customisable Fairness Calibrationcited by 6
  20. 2024Biased AI can Influence Political Decision-Makingcited by 6
  21. 2025Exposing Product Bias in LLM Investment Recommendationcited by 6
  22. 2023A Group Fairness Lens for Large Language Modelscited by 4
  23. 2025Metamorphic Testing for Fairness Evaluation in Large Language Models: Identifying Intersectional Bias in LLaMA and GPTcited by 4
  24. 2025Mind the Language Gap: Automated and Augmented Evaluation of Bias in LLMs for High- and Low-Resource Languagescited by 4
  25. 2023An Image is Worth a Thousand Toxic Words: A Metamorphic Testing Framework for Content Moderation Softwarecited by 3
  26. 2024Eagle: Ethical Dataset Given from Real Interactionscited by 3
  27. 2025AI Will Always Love You: Studying Implicit Biases in Romantic AI Companionscited by 3
  28. 2026AI as Entertainmentcited by 2
  29. 2026Moral Sensitivity in LLMs: A Tiered Evaluation of Contextual Bias via Behavioral Profiling and Mechanistic Interpretabilitycited by 2
  30. 2026Harsher on Male? Evaluating LLMs on Gender-Asymmetric Moral Framing Across Diverse Conflict Scenarioscited by 2
  31. 2026Redirected, Not Removed: Task-Dependent Stereotyping Reveals the Limits of LLM Alignmentscited by 1
  32. 2026Social Bias in LLM-Generated Code: Benchmark and Mitigationcited by 1
  33. 2025Bias Testing and Mitigation in Black Box LLMs using Metamorphic Relationscited by 0
  34. 2026ECHO: An Open Research Platform for Evaluation of Chat, Human Behavior, and Outcomescited by 0
  35. 2026Intersectional Fairness in Large Language Modelscited by 0
  36. 2026Ethics Testing: Proactive Identification of Generative AI System Harmscited by 0
  37. 2026TriEval: A Resource-Efficient Pipeline for LLM Bias, Toxicity, and Truthfulness Assessmentcited by 0
  38. 2026RPAM: A Principled Metric for Evaluating Associations in Language Models with High Predictive Validity in Downstream Outputscited by 0

50 most cited of 52 All 52 (JSON)