# Quantifying System-Level Harms from AI Adoption in Complex Sociotechnical Systems

arXiv: https://arxiv.org/abs/2608.23906 · HTML: https://mlsanity.com/paper/2608.23906/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 55 references that are in the map.
- First posted on arXiv 24 Aug 2026.

## Builds on (most cited first, 7 of 7)

- [Generative artificial intelligence enhances creativity but reduces the diversity of novel content](https://mlsanity.com/paper/2312.00506/) (2023)
- [Sociotechnical Safety Evaluation of Generative AI Systems](https://mlsanity.com/paper/2310.11986/) (2023)
- [From plane crashes to algorithmic harm: applicability of safety engineering frameworks for responsible ML](https://mlsanity.com/paper/2210.03535/) (2022)
- [Current and Near-Term AI as a Potential Existential Risk Factor](https://mlsanity.com/paper/2209.10604/) (2022)
- [From Silos to Systems: Process-Oriented Hazard Analysis for AI Systems](https://mlsanity.com/paper/2410.22526/) (2024)
- [Systematic Hazard Analysis for Frontier AI using STPA](https://mlsanity.com/paper/2506.01782/) (2025)
- [Mind the Gap! Pathways Towards Unifying AI Safety and Ethics Research](https://mlsanity.com/paper/2512.10058/) (2025)

## Abstract (arXiv)

Artificial Intelligence (AI) is increasingly integrated into complex sociotechnical systems, including Critical National Infrastructure (CNI), where harms emerge from interactions between technical, human, and organisational elements. Yet current AI evaluation remains model-centric, offering little insight into how observed behaviours might translate into system-level risk. We propose a framework that links structured hazard analysis, component-level testing, and probabilistic system modelling to bridge this gap. By providing a traceable pathway from model behaviour to system-level outcomes, the framework enables practitioners to answer the "so what?" of AI failures, quantify their systemic impact, and move toward evidence-based and anticipatory governance of AI in complex systems. Applied to the UK's Real Time Gross Settlement (RTGS) system as an illustrative worked example, we derive AI-driven loss scenarios using Systems Theoretic Process Analysis (STPA) and examine adversarial manipulation of LLM-based trading as one such loss scenario. Component-level experiments show that simple adversarial inputs induce measurable behavioural shifts where AI recommendations are followed. Under the component-to-system mapping used here for a financial contagion model, these shifts alter system resilience, increasing bank failures and lowering the threshold at which shocks lead to cascading disruption, particularly under widespread or monopolistic AI adoption.