# Human-centered mechanism design with Democratic AI

arXiv: https://arxiv.org/abs/2201.11441 · HTML: https://mlsanity.com/paper/2201.11441/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 37 references that are in the map.
- First posted on arXiv 27 Jan 2022.

## Builds on (most cited first, 11 of 11)

- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Artificial Intelligence, Values and Alignment](https://mlsanity.com/paper/2001.09768/) (2020)
- [On the Utility of Learning about Humans for Human-AI Coordination](https://mlsanity.com/paper/1910.05789/) (2019)
- [Open Problems in Cooperative AI](https://mlsanity.com/paper/2012.08630/) (2020)
- [Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence](https://mlsanity.com/paper/2007.04068/) (2020)
- [The AI Economist: Improving Equality and Productivity with AI-Driven Tax Policies](https://mlsanity.com/paper/2004.13332/) (2020)
- [A Voting-Based System for Ethical Decision Making](https://mlsanity.com/paper/1709.06692/) (2017)
- [Imitating Interactive Intelligence](https://mlsanity.com/paper/2012.05672/) (2020)
- [Society-in-the-Loop: Programming the Algorithmic Social Contract](https://mlsanity.com/paper/1707.07232/) (2017)
- [Mechanism Design for Social Good](https://mlsanity.com/paper/1810.09832/) (2018)

## Built on by (most cited first, 3 of 3)

- [Law Informs Code: A Legal Informatics Approach to Aligning Artificial Intelligence with Humans](https://mlsanity.com/paper/2209.13020/) (2022)
- [AI Agent Behavioral Science](https://mlsanity.com/paper/2506.06366/) (2025)
- [Calibrating Artificial Guilt: Neurally Grounded Reward Shaping for Prosocial Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2608.04663/) (2026)

## Abstract (arXiv)

Building artificial intelligence (AI) that aligns with human values is an unsolved problem. Here, we developed a human-in-the-loop research pipeline called Democratic AI, in which reinforcement learning is used to design a social mechanism that humans prefer by majority. A large group of humans played an online investment game that involved deciding whether to keep a monetary endowment or to share it with others for collective benefit. Shared revenue was returned to players under two different redistribution mechanisms, one designed by the AI and the other by humans. The AI discovered a mechanism that redressed initial wealth imbalance, sanctioned free riders, and successfully won the majority vote. By optimizing for human preferences, Democratic AI may be a promising method for value-aligned policy innovation.