# Measuring Ethics in AI with AI: A Methodology and Dataset Construction

arXiv: https://arxiv.org/abs/2107.11913 · HTML: https://mlsanity.com/paper/2107.11913/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 30 references that are in the map.
- First posted on arXiv 26 Jul 2021.

## Builds on (most cited first, 12 of 12)

- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Deep Learning in Neural Networks: An Overview](https://mlsanity.com/paper/1404.7828/) (2014)
- [What Does Explainable AI Really Mean? A New Conceptualization of Perspectives](https://mlsanity.com/paper/1710.00794/) (2017)
- [Beyond Fair Pay: Ethical Implications of NLP Crowdsourcing](https://mlsanity.com/paper/2104.10097/) (2021)
- [The AI Index 2021 Annual Report](https://mlsanity.com/paper/2103.06312/) (2021)
- [Building Ethically Bounded AI](https://mlsanity.com/paper/1812.03980/) (2018)
- [Data Science as Political Action: Grounding Data Science in a Politics of Justice](https://mlsanity.com/paper/1811.03435/) (2018)
- [Ethical Considerations in Artificial Intelligence Courses](https://mlsanity.com/paper/1701.07769/) (2017)
- [Measurement in AI Policy: Opportunities and Challenges](https://mlsanity.com/paper/2009.09071/) (2020)
- [On Quantifying and Understanding the Role of Ethics in AI Research: A Historical Account of Flagship Conferences and Journals](https://mlsanity.com/paper/1809.08328/) (2018)
- [Approaching Ethical Guidelines for Data Scientists](https://mlsanity.com/paper/1901.04824/) (2019)
- [Teaching AI, Ethics, Law and Policy](https://mlsanity.com/paper/1904.12470/) (2019)

## Built on by (most cited first, 1 of 1)

- [Fairness Concerns in App Reviews: A Study on AI-based Mobile Apps](https://mlsanity.com/paper/2401.08097/) (2024)

## Abstract (arXiv)

Recently, the use of sound measures and metrics in Artificial Intelligence has become the subject of interest of academia, government, and industry. Efforts towards measuring different phenomena have gained traction in the AI community, as illustrated by the publication of several influential field reports and policy documents. These metrics are designed to help decision takers to inform themselves about the fast-moving and impacting influences of key advances in Artificial Intelligence in general and Machine Learning in particular. In this paper we propose to use such newfound capabilities of AI technologies to augment our AI measuring capabilities. We do so by training a model to classify publications related to ethical issues and concerns. In our methodology we use an expert, manually curated dataset as the training set and then evaluate a large set of research papers. Finally, we highlight the implications of AI metrics, in particular their contribution towards developing trustful and fair AI-based tools and technologies. Keywords: AI Ethics; AI Fairness; AI Measurement. Ethics in Computer Science.