mlsanityData to 7 Oct 2026

Data Ethics Emergency Drill: A Toolbox for Discussing Responsible AI for Industry Teams

arXiv 2403.10438 ↗First posted 15 Mar 2024cs.HC cs.AI cs.CYcited by 3

Not on the map. Why?

Abstract (arXiv)

Researchers urge technology practitioners such as data scientists to consider the impacts and ethical implications of algorithmic decisions. However, unlike programming, statistics, and data management, discussion of ethical implications is rarely included in standard data science training. To begin to address this gap, we designed and tested a toolbox called the data ethics emergency drill (DEED) to help data science teams discuss and reflect on the ethical implications of their work. The DEED is a roleplay of a fictional ethical emergency scenario that is contextually situated in the team's specific workplace and applications. This paper outlines the DEED toolbox and describes three studies carried out with two different data science teams that iteratively shaped its design. Our findings show that practitioners can apply lessons learnt from the roleplay to real-life situations, and how the DEED opened up conversations around ethics and values.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 18 of its 66 references that are in the map. First posted on arXiv 15 Mar 2024.

Builds on

18 of 66 references.

References · 18

  1. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  2. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  3. 2018Datasheets for Datasetscited by 1,371
  4. 2018Model Cards for Model Reportingcited by 1,068
  5. 2018Improving fairness in machine learning systems: What do industry practitioners need?cited by 262
  6. 2018Learning Gender-Neutral Word Embeddingscited by 169
  7. 2019The Ethics of AI Ethics -- An Evaluation of Guidelinescited by 129
  8. 2019Problem Formulation and Fairnesscited by 85
  9. 2023Who Audits the Auditors? Recommendations from a field scan of the algorithmic auditing ecosystemcited by 49
  10. 2022Dislocated Accountabilities in the AI Supply Chain: Modularity and Developers' Notions of Responsibilitycited by 33
  11. 2022The Forgotten Margins of AI Ethicscited by 30
  12. 2022Seeing Like a Toolkit: How Toolkits Envision the Work of AI Ethicscited by 28
6 more
  1. 2021Machine Learning Practices Outside Big Tech: How Resource Constraints Challenge Responsible Developmentcited by 21
  2. 2023`It is currently hodgepodge'': Examining AI/ML Practitioners' Challenges during Co-production of Responsible AI Valuescited by 18
  3. 2023"That's important, but...": How Computer Science Researchers Anticipate Unintended Consequences of Their Research Innovationscited by 8
  4. 2023Fairness Evaluation in Text Classification: Machine Learning Practitioner Perspectives of Individual and Group Fairnesscited by 7
  5. 2019Exploring AI Futures Through Role Playcited by 3
  6. 2023A toolkit of dilemmas: Beyond debiasing and fairness formulas for responsible AI/MLcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Evaluation of AI Ethics Tools in Language Models: A Developers' Perspective Case Studycited by 0
  2. 2026What We Know about Responsible AI Practices in Industry: A Half Decade of Empirical Researchcited by 0
  3. 2026Ethics Training Agents: Facilitating Group-Based Ethics Education with Role-Playing and Discussion for Ethical Reflection and Explorationcited by 0