# Mediating Artificial Intelligence Developments through Negative and Positive Incentives

arXiv: https://arxiv.org/abs/2010.00403 · HTML: https://mlsanity.com/paper/2010.00403/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Oct 2020.

## Built on by (most cited first, 9 of 9)

- [Voluntary safety commitments provide an escape from over-regulation in AI development](https://mlsanity.com/paper/2104.03741/) (2021)
- [Trust AI Regulation? Discerning users are vital to build trust and effective AI regulation](https://mlsanity.com/paper/2403.09510/) (2024)
- [Social physics in the age of artificial intelligence](https://mlsanity.com/paper/2603.16900/) (2026)
- [Evolutionary mechanisms that promote cooperation may not promote social welfare](https://mlsanity.com/paper/2408.05373/) (2024)
- [Both eyes open: Vigilant Incentives help Regulatory Markets improve AI Safety](https://mlsanity.com/paper/2303.03174/) (2023)
- [Understanding Emergent Behaviours in Multi-Agent Systems with Evolutionary Game Theory](https://mlsanity.com/paper/2205.07369/) (2022)
- [Falling Behind Drives Unsafe Development in an Idealised AI Race Experiment](https://mlsanity.com/paper/2607.26034/) (2026)
- [Safe Transformative AI via a Windfall Clause](https://mlsanity.com/paper/2108.09404/) (2021)
- [Evolutionary game theory: the mathematics of evolution and collective behaviours](https://mlsanity.com/paper/2311.14480/) (2023)

## Abstract (arXiv)

The field of Artificial Intelligence (AI) is going through a period of great expectations, introducing a certain level of anxiety in research, business and also policy. This anxiety is further energised by an AI race narrative that makes people believe they might be missing out. Whether real or not, a belief in this narrative may be detrimental as some stake-holders will feel obliged to cut corners on safety precautions, or ignore societal consequences just to "win". Starting from a baseline model that describes a broad class of technology races where winners draw a significant benefit compared to others (such as AI advances, patent race, pharmaceutical technologies), we investigate here how positive (rewards) and negative (punishments) incentives may beneficially influence the outcomes. We uncover conditions in which punishment is either capable of reducing the development speed of unsafe participants or has the capacity to reduce innovation through over-regulation. Alternatively, we show that, in several scenarios, rewarding those that follow safety measures may increase the development speed while ensuring safe choices. Moreover, in {the latter} regimes, rewards do not suffer from the issue of over-regulation as is the case for punishment. Overall, our findings provide valuable insights into the nature and kinds of regulatory actions most suitable to improve safety compliance in the contexts of both smooth and sudden technological shifts.