# Concrete Safety for ML Problems: System Safety for ML Development and Assessment

arXiv: https://arxiv.org/abs/2302.02972 · HTML: https://mlsanity.com/paper/2302.02972/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 58 references that are in the map.
- First posted on arXiv 6 Feb 2023.

## Builds on (most cited first, 13 of 13)

- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing](https://mlsanity.com/paper/2001.00973/) (2020)
- [AI and the Everything in the Whole Wide World Benchmark](https://mlsanity.com/paper/2111.15366/) (2021)
- [Unsolved Problems in ML Safety](https://mlsanity.com/paper/2109.13916/) (2021)
- [Improving fairness in machine learning systems: What do industry practitioners need?](https://mlsanity.com/paper/1812.05239/) (2018)
- [Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support](https://mlsanity.com/paper/2112.05675/) (2021)
- [Accountability in an Algorithmic Society: Relationality, Responsibility, and Robustness in Machine Learning](https://mlsanity.com/paper/2202.05338/) (2022)
- [Participatory Problem Formulation for Fairer Machine Learning Through Community Based System Dynamics](https://mlsanity.com/paper/2005.07572/) (2020)
- [From plane crashes to algorithmic harm: applicability of safety engineering frameworks for responsible ML](https://mlsanity.com/paper/2210.03535/) (2022)
- [This Thing Called Fairness: Disciplinary Confusion Realizing a Value in Technology](https://mlsanity.com/paper/1909.11869/) (2019)
- [A Hierarchy of Limitations in Machine Learning](https://mlsanity.com/paper/2002.05193/) (2020)
- [Extending the Machine Learning Abstraction Boundary: A Complex Systems Approach to Incorporate Societal Context](https://mlsanity.com/paper/2006.09663/) (2020)
- [System Safety and Artificial Intelligence](https://mlsanity.com/paper/2202.09292/) (2022)

## Built on by (most cited first, 2 of 2)

- [On the Limitations of Compute Thresholds as a Governance Strategy](https://mlsanity.com/paper/2407.05694/) (2024)
- [Unsafe at any AUC: Unlearned Lessons from Sociotechnical Disasters for Responsible AI](https://mlsanity.com/paper/2607.14353/) (2026)

## Abstract (arXiv)

Many stakeholders struggle to make reliances on ML-driven systems due to the risk of harm these systems may cause. Concerns of trustworthiness, unintended social harms, and unacceptable social and ethical violations undermine the promise of ML advancements. Moreover, such risks in complex ML-driven systems present a special challenge as they are often difficult to foresee, arising over periods of time, across populations, and at scale. These risks often arise not from poor ML development decisions or low performance directly but rather emerge through the interactions amongst ML development choices, the context of model use, environmental factors, and the effects of a model on its target. Systems safety engineering is an established discipline with a proven track record of identifying and managing risks even in high-complexity sociotechnical systems. In this work, we apply a state-of-the-art systems safety approach to concrete applications of ML with notable social and ethical risks to demonstrate a systematic means for meeting the assurance requirements needed to argue for safe and trustworthy ML in sociotechnical systems.