mlsanityData to 14 Oct 2026

Step by Step to Fairness: Attributing Societal Bias in Task-oriented Dialogue Systems

arXiv 2311.06513 ↗First posted 11 Nov 2023cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent works have shown considerable improvements in task-oriented dialogue (TOD) systems by utilizing pretrained large language models (LLMs) in an end-to-end manner. However, the biased behavior of each component in a TOD system and the error propagation issue in the end-to-end framework can lead to seriously biased TOD responses. Existing works of fairness only focus on the total bias of a system. In this paper, we propose a diagnosis method to attribute bias to each component of a TOD system. With the proposed attribution method, we can gain a deeper understanding of the sources of bias. Additionally, researchers can mitigate biased model behavior at a more granular level. We conduct experiments to attribute the TOD system's bias toward three demographic axes: gender, age, and race. Experimental results show that the bias of a TOD system usually comes from the response generation model.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 32 of its 41 references that are in the map. First posted on arXiv 11 Nov 2023.

Builds on

32 of 41 references.

References · 32

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  2. 2016Equality of Opportunity in Supervised Learningcited by 2,007
  3. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 1,006
  4. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 924
  5. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 541
  6. 2019Learning the Difference that Makes a Difference with Counterfactually-Augmented Datacited by 439
  7. 2017Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraintscited by 420
  8. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 412
  9. 2019On Measuring Social Biases in Sentence Encoderscited by 398
  10. 2020A Simple Language Model for Task-Oriented Dialoguecited by 332
  11. 2021Towards Understanding and Mitigating Social Biases in Language Modelscited by 296
  12. 2019Measuring Bias in Contextualized Word Representationscited by 276
20 more
  1. 2020MultiWOZ 2.2 : A Dialogue Dataset with Additional Annotation Corrections and State Tracking Baselinescited by 204
  2. 2021Societal Biases in Language Generation: Progress and Challengescited by 202
  3. 2019Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphologycited by 200
  4. 2019Queens are Powerful too: Mitigating Gender Bias in Dialogue Generationcited by 152
  5. 2019Reducing Sentiment Bias in Language Models via Counterfactual Evaluationcited by 148
  6. 2019Taskmaster-1: Toward a Realistic and Diverse Dialog Datasetcited by 140
  7. 2021Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Surveycited by 130
  8. 2020Towards Controllable Biases in Language Generationcited by 112
  9. 2019Does Gender Matter? Towards Fairness in Dialogue Systemscited by 89
  10. 2021Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metricscited by 83
  11. 2018Conversational AI: The Science Behind the Alexa Prizecited by 63
  12. 2021Zero-Shot Dialogue State Tracking via Cross-Task Transfercited by 52
  13. 2018Microsoft Dialogue Challenge: Building End-to-End Task-Completion Dialogue Systemscited by 47
  14. 2020Mitigating Gender Bias for Neural Dialogue Generation with Adversarial Learningcited by 44
  15. 2020CoCo: Controllable Counterfactuals for Evaluating Dialogue State Trackerscited by 43
  16. 2021SGD-X: A Benchmark for Robust Generalization in Schema-Guided Dialogue Systemscited by 34
  17. 2021Annotation Inconsistency and Entity Bias in MultiWOZcited by 30
  18. 2022How sensitive are translation systems to extra contexts? Mitigating gender bias in Neural Machine Translation models through relevant contextscited by 10
  19. 2021Know Thy Strengths: Comprehensive Dialogue State Tracking Diagnosticscited by 9
  20. 2021Teaching Models new APIs: Domain-Agnostic Simulators for Task Oriented Dialoguecited by 6

Built on by

Not cited yet.