# Non-Determinism and the Lawlessness of Machine Learning Code

arXiv: https://arxiv.org/abs/2206.11834 · HTML: https://mlsanity.com/paper/2206.11834/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 50 references that are in the map.
- First posted on arXiv 23 Jun 2022.

## Builds on (most cited first, 7 of 7)

- [Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty](https://mlsanity.com/paper/2011.07586/) (2020)
- [A Step Toward Quantifying Independently Reproducible Machine Learning Research](https://mlsanity.com/paper/1909.06674/) (2019)
- [Accountability in an Algorithmic Society: Relationality, Responsibility, and Robustness in Machine Learning](https://mlsanity.com/paper/2202.05338/) (2022)
- [Hyperparameter Optimization Is Deceiving Us, and How to Stop It](https://mlsanity.com/paper/2102.03034/) (2021)
- [An Uncommon Task: Participatory Design in Legal AI](https://mlsanity.com/paper/2203.06246/) (2022)
- [Model Selection's Disparate Impact in Real-World Deep Learning Applications](https://mlsanity.com/paper/2104.00606/) (2021)
- [Accuracy-Efficiency Trade-Offs and Accountability in Distributed ML Systems](https://mlsanity.com/paper/2007.02203/) (2020)

## Built on by (most cited first, 7 of 7)

- [METAL: Metamorphic Testing Framework for Analyzing Large-Language Model Qualities](https://mlsanity.com/paper/2312.06056/) (2023)
- [Machine Unlearning Doesn't Do What You Think: Lessons for Generative AI Policy and Research](https://mlsanity.com/paper/2412.06966/) (2024)
- [Systemizing Multiplicity: The Curious Case of Arbitrariness in Machine Learning](https://mlsanity.com/paper/2501.14959/) (2025)
- [Mixtera: A Data Plane for Foundation Model Training](https://mlsanity.com/paper/2502.19790/) (2025)
- [Using predictive multiplicity to measure individual performance within the AI Act](https://mlsanity.com/paper/2602.11944/) (2026)
- [Data as a Lever: A Neighbouring Datasets Perspective on Predictive Multiplicity](https://mlsanity.com/paper/2510.21303/) (2025)
- [A Model-Driven Lossless Compression Algorithm Resistant to Mismatch](https://mlsanity.com/paper/2601.17684/) (2026)

## Abstract (arXiv)

Legal literature on machine learning (ML) tends to focus on harms, and thus tends to reason about individual model outcomes and summary error rates. This focus has masked important aspects of ML that are rooted in its reliance on randomness -- namely, stochasticity and non-determinism. While some recent work has begun to reason about the relationship between stochasticity and arbitrariness in legal contexts, the role of non-determinism more broadly remains unexamined. In this paper, we clarify the overlap and differences between these two concepts, and show that the effects of non-determinism, and consequently its implications for the law, become clearer from the perspective of reasoning about ML outputs as distributions over possible outcomes. This distributional viewpoint accounts for randomness by emphasizing the possible outcomes of ML. Importantly, this type of reasoning is not exclusive with current legal reasoning; it complements (and in fact can strengthen) analyses concerning individual, concrete outcomes for specific automated decisions. By illuminating the important role of non-determinism, we demonstrate that ML code falls outside of the cyberlaw frame of treating ``code as law,'' as this frame assumes that code is deterministic. We conclude with a brief discussion of what work ML can do to constrain the potentially harm-inducing effects of non-determinism, and we indicate where the law must do work to bridge the gap between its current individual-outcome focus and the distributional approach that we recommend.