# ESR: Ethics and Society Review of Artificial Intelligence Research

arXiv: https://arxiv.org/abs/2106.11521 · HTML: https://mlsanity.com/paper/2106.11521/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jun 2021.

## Built on by (most cited first, 6 of 6)

- [Jury Learning: Integrating Dissenting Voices into Machine Learning Models](https://mlsanity.com/paper/2202.02950/) (2022)
- [AI Ethics Statements -- Analysis and lessons learnt from NeurIPS Broader Impact Statements](https://mlsanity.com/paper/2111.01705/) (2021)
- [An animated picture says at least a thousand words: Selecting Gif-based Replies in Multimodal Dialog](https://mlsanity.com/paper/2109.12212/) (2021)
- [Thorny Roses: Investigating the Dual Use Dilemma in Natural Language Processing](https://mlsanity.com/paper/2304.08315/) (2023)
- [A Comprehensive Survey and Classification of Evaluation Criteria for Trustworthy Artificial Intelligence](https://mlsanity.com/paper/2410.17281/) (2024)
- [Societal AI Research Has Become Less Interdisciplinary](https://mlsanity.com/paper/2506.08738/) (2025)

## Abstract (arXiv)

Artificial intelligence (AI) research is routinely criticized for its real and potential impacts on society, and we lack adequate institutional responses to this criticism and to the responsibility that it reflects. AI research often falls outside the purview of existing feedback mechanisms such as the Institutional Review Board (IRB), which are designed to evaluate harms to human subjects rather than harms to human society. In response, we have developed the Ethics and Society Review board (ESR), a feedback panel that works with researchers to mitigate negative ethical and societal aspects of AI research. The ESR's main insight is to serve as a requirement for funding: researchers cannot receive grant funding from a major AI funding program at our university until the researchers complete the ESR process for the proposal. In this article, we describe the ESR as we have designed and run it over its first year across 41 proposals. We analyze aggregate ESR feedback on these proposals, finding that the panel most commonly identifies issues of harms to minority groups, inclusion of diverse stakeholders in the research plan, dual use, and representation in data. Surveys and interviews of researchers who interacted with the ESR found that 58% felt that it had influenced the design of their research project, 100% are willing to continue submitting future projects to the ESR, and that they sought additional scaffolding for reasoning through ethics and society issues.