# Choices, Risks, and Reward Reports: Charting Public Policy for Reinforcement Learning Systems

arXiv: https://arxiv.org/abs/2202.05716 · HTML: https://mlsanity.com/paper/2202.05716/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 34 references that are in the map.
- First posted on arXiv 11 Feb 2022.

## Builds on (most cited first, 4 of 4)

- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [Driving Through Ghosts: Behavioral Cloning with False Positives](https://mlsanity.com/paper/2008.12969/) (2020)
- [On Identification of Distribution Grids](https://mlsanity.com/paper/1711.01526/) (2017)

## Built on by (most cited first, 12 of 12)

- [Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2504.12501/) (2025)
- [The History and Risks of Reinforcement Learning and Human Feedback](https://mlsanity.com/paper/2310.13595/) (2023)
- [The Alignment Ceiling: Objective Mismatch in Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2311.00168/) (2023)
- [Reward Reports for Reinforcement Learning](https://mlsanity.com/paper/2204.10817/) (2022)
- [Accounting for AI and Users Shaping One Another: The Role of Mathematical Models](https://mlsanity.com/paper/2404.12366/) (2024)
- [Designing Fiduciary Artificial Intelligence](https://mlsanity.com/paper/2308.02435/) (2023)
- [Actionable Guidance for High-Consequence AI Risk Management: Towards Standards Addressing AI Catastrophic Risks](https://mlsanity.com/paper/2206.08966/) (2022)
- [Dynamic Documentation for AI Systems](https://mlsanity.com/paper/2303.10854/) (2023)
- [Incorporating Quality of Life in Climate Adaptation Planning via Reinforcement Learning](https://mlsanity.com/paper/2511.03238/) (2025)
- [Climate Adaptation with Reinforcement Learning: Economic vs. Quality of Life Adaptation Pathways](https://mlsanity.com/paper/2511.03243/) (2025)
- [Learning long term climate-resilient transport adaptation pathways under direct and indirect flood impacts using reinforcement learning](https://mlsanity.com/paper/2601.18586/) (2026)
- [Artificial Intelligence for Climate Adaptation: Reinforcement Learning for Climate Change-Resilient Transport](https://mlsanity.com/paper/2603.06278/) (2026)

## Abstract (arXiv)

In the long term, reinforcement learning (RL) is considered by many AI theorists to be the most promising path to artificial general intelligence. This places RL practitioners in a position to design systems that have never existed before and lack prior documentation in law and policy. Public agencies could intervene on complex dynamics that were previously too opaque to deliberate about, and long-held policy ambitions would finally be made tractable. In this whitepaper we illustrate this potential and how it might be technically enacted in the domains of energy infrastructure, social media recommender systems, and transportation. Alongside these unprecedented interventions come new forms of risk that exacerbate the harms already generated by standard machine learning tools. We correspondingly present a new typology of risks arising from RL design choices, falling under four categories: scoping the horizon, defining rewards, pruning information, and training multiple agents. Rather than allowing RL systems to unilaterally reshape human domains, policymakers need new mechanisms for the rule of reason, foreseeability, and interoperability that match the risks these systems pose. We argue that criteria for these choices may be drawn from emerging subfields within antitrust, tort, and administrative law. It will then be possible for courts, federal and state agencies, and non-governmental organizations to play more active roles in RL specification and evaluation. Building on the "model cards" and "datasheets" frameworks proposed by Mitchell et al. and Gebru et al., we argue the need for Reward Reports for AI systems. Reward Reports are living documents for proposed RL deployments that demarcate design choices.