# Exploring How Machine Learning Practitioners (Try To) Use Fairness Toolkits

arXiv: https://arxiv.org/abs/2205.06922 · HTML: https://mlsanity.com/paper/2205.06922/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 122 references that are in the map.
- First posted on arXiv 13 May 2022.

## Builds on (most cited first, 30 of 40)

- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [Fair prediction with disparate impact: A study of bias in recidivism prediction instruments](https://mlsanity.com/paper/1610.07524/) (2016)
- [Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness](https://mlsanity.com/paper/1711.05144/) (2017)
- [Improving fairness in machine learning systems: What do industry practitioners need?](https://mlsanity.com/paper/1812.05239/) (2018)
- [Data and its (dis)contents: A survey of dataset development and use in machine learning research](https://mlsanity.com/paper/2012.05345/) (2020)
- [A comparative study of fairness-enhancing interventions in machine learning](https://mlsanity.com/paper/1802.04422/) (2018)
- [The What-If Tool: Interactive Probing of Machine Learning Models](https://mlsanity.com/paper/1907.04135/) (2019)
- [Aequitas: A Bias and Fairness Audit Toolkit](https://mlsanity.com/paper/1811.05577/) (2018)
- [Re-imagining Algorithmic Fairness in India and Beyond](https://mlsanity.com/paper/2101.09995/) (2021)
- [The Dataset Nutrition Label: A Framework To Drive Higher Data Quality Standards](https://mlsanity.com/paper/1805.03677/) (2018)
- [FairVis: Visual Analytics for Discovering Intersectional Bias in Machine Learning](https://mlsanity.com/paper/1904.05419/) (2019)
- [Problem Formulation and Fairness](https://mlsanity.com/paper/1901.02547/) (2019)
- [Do Datasets Have Politics? Disciplinary Values in Computer Vision Dataset Development](https://mlsanity.com/paper/2108.04308/) (2021)
- [A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores](https://mlsanity.com/paper/2002.08035/) (2020)
- [How do Data Science Workers Collaborate? Roles, Workflows, and Tools](https://mlsanity.com/paper/2001.06684/) (2020)
- [Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support](https://mlsanity.com/paper/2112.05675/) (2021)
- [FairSight: Visual Analytics for Fairness in Decision Making](https://mlsanity.com/paper/1908.00176/) (2019)
- [Mathematical Notions vs. Human Perception of Fairness: A Descriptive Approach to Fairness for Machine Learning](https://mlsanity.com/paper/1902.04783/) (2019)
- ["What We Can't Measure, We Can't Understand": Challenges to Demographic Data Procurement in the Pursuit of Fairness](https://mlsanity.com/paper/2011.02282/) (2020)
- [Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors](https://mlsanity.com/paper/2105.02980/) (2021)
- [Factors Influencing Perceived Fairness in Algorithmic Decision-Making: Algorithm Outcomes, Development Procedures, and Individual Differences](https://mlsanity.com/paper/2001.09604/) (2020)
- [Where Responsible AI meets Reality: Practitioner Perspectives on Enablers for shifting Organizational Practices](https://mlsanity.com/paper/2006.12358/) (2020)
- [Human Perceptions of Fairness in Algorithmic Decision Making: A Case Study of Criminal Risk Prediction](https://mlsanity.com/paper/1802.09548/) (2018)
- [Reviewable Automated Decision-Making: A Framework for Accountable Algorithmic Systems](https://mlsanity.com/paper/2102.04201/) (2021)
- [Discovering and Validating AI Errors With Crowdsourced Failure Reports](https://mlsanity.com/paper/2109.11690/) (2021)
- [Between Subjectivity and Imposition: Power Dynamics in Data Annotation for Computer Vision](https://mlsanity.com/paper/2007.14886/) (2020)
- [Themis-ml: A Fairness-aware Machine Learning Interface for End-to-end Discrimination Discovery and Mitigation](https://mlsanity.com/paper/1710.06921/) (2017)

## Built on by (most cited first, 30 of 34)

- [Design Principles for Generative AI Applications](https://mlsanity.com/paper/2401.14484/) (2024)
- [Understanding Practices, Challenges, and Opportunities for User-Engaged Algorithm Auditing in Industry Practice](https://mlsanity.com/paper/2210.03709/) (2022)
- [LLM-Driven Robots Risk Enacting Discrimination, Violence, and Unlawful Actions](https://mlsanity.com/paper/2406.08824/) (2024)
- [Real Risks of Fake Data: Synthetic Data, Diversity-Washing and Consent Circumvention](https://mlsanity.com/paper/2405.01820/) (2024)
- [Seamful XAI: Operationalizing Seamful Design in Explainable AI](https://mlsanity.com/paper/2211.06753/) (2022)
- [Designerly Understanding: Information Needs for Model Transparency to Support Design Ideation for AI-Powered User Experience](https://mlsanity.com/paper/2302.10395/) (2023)
- [Can Fairness be Automated? Guidelines and Opportunities for Fairness-aware AutoML](https://mlsanity.com/paper/2303.08485/) (2023)
- [Out of Context: Investigating the Bias and Fairness Concerns of "Artificial Intelligence as a Service"](https://mlsanity.com/paper/2302.01448/) (2023)
- [Investigating Practices and Opportunities for Cross-functional Collaboration around AI Fairness in Industry Practice](https://mlsanity.com/paper/2306.06542/) (2023)
- [A.I. Robustness: a Human-Centered Perspective on Technological Challenges and Opportunities](https://mlsanity.com/paper/2210.08906/) (2022)
- [PersonaTeaming: Exploring How Introducing Personas Can Improve Automated AI Red-Teaming](https://mlsanity.com/paper/2509.03728/) (2025)
- [Fairness Evaluation in Text Classification: Machine Learning Practitioner Perspectives of Individual and Group Fairness](https://mlsanity.com/paper/2303.00673/) (2023)
- [From Paper to Card: Transforming Design Implications with Generative AI](https://mlsanity.com/paper/2403.08137/) (2024)
- [Navigating Fairness Measures and Trade-Offs](https://mlsanity.com/paper/2307.08484/) (2023)
- [A Catalog of Fairness-Aware Practices in Machine Learning Engineering](https://mlsanity.com/paper/2408.16683/) (2024)
- [MIRAGE: Multi-model Interface for Reviewing and Auditing Generative Text-to-Image AI](https://mlsanity.com/paper/2503.19252/) (2025)
- [SuperNOVA: Design Strategies and Opportunities for Interactive Visualization in Computational Notebooks](https://mlsanity.com/paper/2305.03039/) (2023)
- [Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems](https://mlsanity.com/paper/2506.04482/) (2025)
- [Addressing Pitfalls in Auditing Practices of Automatic Speech Recognition Technologies: A Case Study of People with Aphasia](https://mlsanity.com/paper/2506.08846/) (2025)
- [A Taxonomy of Challenges to Curating Fair Datasets](https://mlsanity.com/paper/2406.06407/) (2024)
- [Articulation Work and Tinkering for Fairness in Machine Learning](https://mlsanity.com/paper/2407.16496/) (2024)
- [Talking About the Assumption in the Room](https://mlsanity.com/paper/2502.13268/) (2025)
- [Trustworthy, responsible, ethical AI in manufacturing and supply chains: synthesis and emerging research questions](https://mlsanity.com/paper/2305.11581/) (2023)
- [Navigating Fairness: Practitioners' Understanding, Challenges, and Strategies in AI/ML Development](https://mlsanity.com/paper/2403.15481/) (2024)
- [From Expectation to Habit: Why Do Software Practitioners Adopt Fairness Toolkits?](https://mlsanity.com/paper/2412.13846/) (2024)
- [AI LEGO: Scaffolding Cross-Functional Collaboration in Industrial Responsible AI Practices during Early Design Stages](https://mlsanity.com/paper/2505.10300/) (2025)
- [Measurement as Bricolage: Examining How Data Scientists Construct Target Variables for Predictive Modeling Tasks](https://mlsanity.com/paper/2507.02819/) (2025)
- [From Reflection to Repair: A Scoping Review of Dataset Documentation Tools](https://mlsanity.com/paper/2602.15968/) (2026)
- [Assessing Perceived Fairness from Machine Learning Developer's Perspective](https://mlsanity.com/paper/2304.03745/) (2023)
- [Data Preparation for Fairness-Performance Trade-Offs: A Practitioner-Friendly Alternative?](https://mlsanity.com/paper/2412.15920/) (2024)

## Abstract (arXiv)

Recent years have seen the development of many open-source ML fairness toolkits aimed at helping ML practitioners assess and address unfairness in their systems. However, there has been little research investigating how ML practitioners actually use these toolkits in practice. In this paper, we conducted the first in-depth empirical exploration of how industry practitioners (try to) work with existing fairness toolkits. In particular, we conducted think-aloud interviews to understand how participants learn about and use fairness toolkits, and explored the generality of our findings through an anonymous online survey. We identified several opportunities for fairness toolkits to better address practitioner needs and scaffold them in using toolkits effectively and responsibly. Based on these findings, we highlight implications for the design of future open-source fairness toolkits that can support practitioners in better contextualizing, communicating, and collaborating around ML fairness efforts.