# Diversity and Inclusion Metrics in Subset Selection

arXiv: https://arxiv.org/abs/2002.03256 · HTML: https://mlsanity.com/paper/2002.03256/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Feb 2020.

## Built on by (most cited first, 27 of 27)

- [The Vendi Score: A Diversity Evaluation Metric for Machine Learning](https://mlsanity.com/paper/2210.02410/) (2022)
- [DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Models](https://mlsanity.com/paper/2202.04053/) (2022)
- [Gender Bias in Machine Translation](https://mlsanity.com/paper/2104.06001/) (2021)
- [Survey of Bias In Text-to-Image Generation: Definition, Evaluation, and Mitigation](https://mlsanity.com/paper/2404.01030/) (2024)
- [SERENGETI: Massively Multilingual Language Models for Africa](https://mlsanity.com/paper/2212.10785/) (2022)
- [GenderBias-VL: Benchmarking Gender Bias in Vision Language Models via Counterfactual Probing](https://mlsanity.com/paper/2407.00600/) (2024)
- [Underspecification in Scene Description-to-Depiction Tasks](https://mlsanity.com/paper/2210.05815/) (2022)
- [A Friendly Face: Do Text-to-Image Systems Rely on Stereotypes when the Input is Under-Specified?](https://mlsanity.com/paper/2302.07159/) (2023)
- [Listening to Affected Communities to Define Extreme Speech: Dataset and Experiments](https://mlsanity.com/paper/2203.11764/) (2022)
- [Consensus and Subjectivity of Skin Tone Annotation for ML Fairness](https://mlsanity.com/paper/2305.09073/) (2023)
- [Bias in Automated Speaker Recognition](https://mlsanity.com/paper/2201.09486/) (2022)
- [Measuring Data](https://mlsanity.com/paper/2212.05129/) (2022)
- [Measuring Hidden Bias within Face Recognition via Racial Phenotypes](https://mlsanity.com/paper/2110.09839/) (2021)
- [Robots Enact Malignant Stereotypes](https://mlsanity.com/paper/2207.11569/) (2022)
- [Justice in Misinformation Detection Systems: An Analysis of Algorithms, Stakeholders, and Potential Harms](https://mlsanity.com/paper/2204.13568/) (2022)
- [Racial Bias within Face Recognition: A Survey](https://mlsanity.com/paper/2305.00817/) (2023)
- [Tiny, always-on and fragile: Bias propagation through design choices in on-device machine learning workflows](https://mlsanity.com/paper/2201.07677/) (2022)
- [RADio -- Rank-Aware Divergence Metrics to Measure Normative Diversity in News Recommendations](https://mlsanity.com/paper/2209.13520/) (2022)
- [Social Norm Bias: Residual Harms of Fairness-Aware Algorithms](https://mlsanity.com/paper/2108.11056/) (2021)
- [Data Representativeness in Accessibility Datasets: A Meta-Analysis](https://mlsanity.com/paper/2207.08037/) (2022)
- [AI and Blackness: Towards moving beyond bias and representation](https://mlsanity.com/paper/2111.03687/) (2021)
- [Challenges and Solutions in AI for All](https://mlsanity.com/paper/2307.10600/) (2023)
- [Deep Learning for Bias Detection: From Inception to Deployment](https://mlsanity.com/paper/2110.15728/) (2021)
- [Designing Data: Proactive Data Collection and Iteration for Machine Learning](https://mlsanity.com/paper/2301.10319/) (2023)
- [Towards Equitable Agile Research and Development of AI and Robotics](https://mlsanity.com/paper/2402.08242/) (2024)
- [Diversity and Inclusion Index with Networks and Similarity: Analysis and its Application](https://mlsanity.com/paper/2410.05668/) (2024)
- [Text-to-Image Representativity Fairness Evaluation Framework](https://mlsanity.com/paper/2410.14201/) (2024)

## Abstract (arXiv)

The ethical concept of fairness has recently been applied in machine learning (ML) settings to describe a wide range of constraints and objectives. When considering the relevance of ethical concepts to subset selection problems, the concepts of diversity and inclusion are additionally applicable in order to create outputs that account for social power and access differentials. We introduce metrics based on these concepts, which can be applied together, separately, and in tandem with additional fairness constraints. Results from human subject experiments lend support to the proposed criteria. Social choice methods can additionally be leveraged to aggregate and choose preferable sets, and we detail how these may be applied.