# AI Assurance in UK Defence: Challenges in Operationalising JSP 936

arXiv: https://arxiv.org/abs/2606.09414 · HTML: https://mlsanity.com/paper/2606.09414/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 120 references that are in the map.
- First posted on arXiv 8 Jun 2026.

## Builds on (most cited first, 29 of 29)

- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Inductive Biases for Deep Learning of Higher-Level Cognition](https://mlsanity.com/paper/2011.15091/) (2020)
- [Multi-Agent Risks from Advanced AI](https://mlsanity.com/paper/2502.14143/) (2025)
- [Quantifying Aleatoric and Epistemic Uncertainty in Machine Learning: Are Conditional Entropy and Mutual Information Appropriate Measures?](https://mlsanity.com/paper/2209.03302/) (2022)
- [Red-Teaming LLM Multi-Agent Systems via Communication Attacks](https://mlsanity.com/paper/2502.14847/) (2025)
- [Fairness And Bias in Artificial Intelligence: A Brief Survey of Sources, Impacts, And Mitigation Strategies](https://mlsanity.com/paper/2304.07683/) (2023)
- [Emergent Abilities in Large Language Models: A Survey](https://mlsanity.com/paper/2503.05788/) (2025)
- [On the Quest for Effectiveness in Human Oversight: Interdisciplinary Perspectives](https://mlsanity.com/paper/2404.04059/) (2024)
- [Risk Analysis Techniques for Governed LLM-based Multi-Agent Systems](https://mlsanity.com/paper/2508.05687/) (2025)
- [Failure Modes in Machine Learning Systems](https://mlsanity.com/paper/1911.11034/) (2019)
- [Requirements Engineering for Artificial Intelligence Systems: A Systematic Mapping Study](https://mlsanity.com/paper/2212.10693/) (2022)
- [The BIG Argument for AI Safety Cases](https://mlsanity.com/paper/2503.11705/) (2025)
- [Failure Modes in LLM Systems: A System-Level Taxonomy for Reliable AI Applications](https://mlsanity.com/paper/2511.19933/) (2025)
- [Guidance on the Safety Assurance of Autonomous Systems in Complex Environments (SACE)](https://mlsanity.com/paper/2208.00853/) (2022)
- [From Silos to Systems: Process-Oriented Hazard Analysis for AI Systems](https://mlsanity.com/paper/2410.22526/) (2024)
- [Status Quo and Problems of Requirements Engineering for Machine Learning: Results from an International Survey](https://mlsanity.com/paper/2310.06726/) (2023)
- [Emergence in Multi-Agent Systems: A Safety Perspective](https://mlsanity.com/paper/2408.04514/) (2024)
- [How Mature is Requirements Engineering for AI-based Systems? A Systematic Mapping Study on Practices, Challenges, and Future Research Directions](https://mlsanity.com/paper/2409.07192/) (2024)
- [AI Safety vs. AI Security: Demystifying the Distinction and Boundaries](https://mlsanity.com/paper/2506.18932/) (2025)
- [Trustworthy Artificial Intelligence in the Context of Metrology](https://mlsanity.com/paper/2406.10117/) (2024)
- [Redefining Safety for Autonomous Vehicles](https://mlsanity.com/paper/2404.16768/) (2024)
- [Lessons from the Cruise Robotaxi Pedestrian Dragging Mishap](https://mlsanity.com/paper/2406.05281/) (2024)
- [A Black Swan Hypothesis: The Role of Human Irrationality in AI Safety](https://mlsanity.com/paper/2407.18422/) (2024)
- [Formalization of Operational Domain and Operational Design Domain for Automated Vehicles](https://mlsanity.com/paper/2408.14481/) (2024)
- [Out-of-Distribution Detection for Safety Assurance of AI and Autonomous Systems](https://mlsanity.com/paper/2510.21254/) (2025)
- [STEAM & MoSAFE: SOTIF Error-and-Failure Model & Analysis for AI-Enabled Driving Automation](https://mlsanity.com/paper/2312.09559/) (2023)
- [Building Trust: Foundations of Security, Safety and Transparency in AI](https://mlsanity.com/paper/2411.12275/) (2024)
- [Agile Management for Machine Learning: A Systematic Mapping Study](https://mlsanity.com/paper/2506.20759/) (2025)

## Abstract (arXiv)

This report examines practical challenges in operationalising JSP 936 Part 1 for AI assurance in UK Defence. Using a structured interpretive review of the directive's requirements, the analysis identifies eight thematic challenge areas adequacy of evidence and argument, management of human interaction with AI, definition of the operational environment, integration of AI within systems of systems, assessment and maintenance of AI performance, analysis of safety and security, measurement of ethicality, and mitigation of the inherent complexities of AI. The report argues that JSP 936 provides a useful governance basis, but that implementation depends on unresolved technical, organisational, and assurance questions. These challenges stem from the socio-technical nature of AI-enabled systems, uncertainty in real-world deployment contexts, limitations in current assurance methodologies, and tensions between performance, safety, human oversight, security, and ethical acceptability. The report identifies areas where further methods, guidance, and organisational capability are needed for the ambitious, safe, and responsible adoption of AI across Defence. This is consistent with MOD's own framing of JSP 936 as requiring iterative implementation and supporting guidance.