# Investigating Algorithm Review Boards for Organizational Responsible Artificial Intelligence Governance

arXiv: https://arxiv.org/abs/2402.01691 · HTML: https://mlsanity.com/paper/2402.01691/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 81 references that are in the map.
- First posted on arXiv 23 Jan 2024.

## Builds on (most cited first, 4 of 4)

- [Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence](https://mlsanity.com/paper/2007.04068/) (2020)
- [Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database](https://mlsanity.com/paper/2011.08512/) (2020)
- [How to design an AI ethics board](https://mlsanity.com/paper/2304.07249/) (2023)
- [Prioritizing Policies for Furthering Responsible Artificial Intelligence in the United States](https://mlsanity.com/paper/2212.00740/) (2022)

## Built on by (most cited first, 3 of 3)

- [SHARP: Social Harm Analysis via Risk Profiles for Measuring Inequities in Large Language Models](https://mlsanity.com/paper/2601.21235/) (2026)
- [What We Know about Responsible AI Practices in Industry: A Half Decade of Empirical Research](https://mlsanity.com/paper/2608.10431/) (2026)
- [LAAF: A Layered Accountability Architecture Framework for LLM Applications](https://mlsanity.com/paper/2608.27102/) (2026)

## Abstract (arXiv)

Organizations including companies, nonprofits, governments, and academic institutions are increasingly developing, deploying, and utilizing artificial intelligence (AI) tools. Responsible AI (RAI) governance approaches at organizations have emerged as important mechanisms to address potential AI risks and harms. In this work, we interviewed 17 technical contributors across organization types (Academic, Government, Industry, Nonprofit) and sectors (Finance, Health, Tech, Other) about their experiences with internal RAI governance. Our findings illuminated the variety of organizational definitions of RAI and accompanying internal governance approaches. We summarized the first detailed findings on algorithm review boards (ARBs) and similar review committees in practice, including their membership, scope, and measures of success. We confirmed known robust model governance in finance sectors and revealed extensive algorithm and AI governance with ARB-like review boards in health sectors. Our findings contradict the idea that Institutional Review Boards alone are sufficient for algorithm governance and posit that ARBs are among the more impactful internal RAI governance approaches. Our results suggest that integration with existing internal regulatory approaches and leadership buy-in are among the most important attributes for success and that financial tensions are the greatest challenge to effective organizational RAI. We make a variety of suggestions for how organizational partners can learn from these findings when building their own internal RAI frameworks. We outline future directions for developing and measuring effectiveness of ARBs and other internal RAI governance approaches.