# AI Forensics Across White-, Grey-, and Black-Box Access: A Process Model and Research Agenda for Post-Incident Investigation of AI Systems

arXiv: https://arxiv.org/abs/2608.03520 · HTML: https://mlsanity.com/paper/2608.03520/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 24 references that are in the map.
- First posted on arXiv 4 Aug 2026.

## Builds on (most cited first, 8 of 8)

- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Stealing Machine Learning Models via Prediction APIs](https://mlsanity.com/paper/1609.02943/) (2016)
- [Towards Observability for Production Machine Learning Pipelines](https://mlsanity.com/paper/2108.13557/) (2021)
- [A Comprehensive Analysis of the Role of Artificial Intelligence and Machine Learning in Modern Digital Forensics and Incident Response](https://mlsanity.com/paper/2309.07064/) (2023)
- [Founding The Domain of AI Forensics](https://mlsanity.com/paper/1912.06497/) (2019)
- [Foundations for Agentic AI Investigations from the Forensic Analysis of OpenClaw](https://mlsanity.com/paper/2604.05589/) (2026)
- [Towards AI Forensics: Did the Artificial Intelligence System Do It?](https://mlsanity.com/paper/2005.13635/) (2020)

## Abstract (arXiv)

AI systems are increasingly involved in decisions and actions that may later require investigation. When an AI related incident occurs, investigators need to reconstruct what the system did, why it behaved that way, and which part of the system or supply chain contributed to the outcome. Existing work on AI forensics remains fragmented, often focusing on a specific system type, artifact, or analysis technique. This paper argues that investigator access is a useful starting point for organizing the field. We distinguish white box, grey box, and black box access and show how each access level changes what can be collected, preserved, analyzed, and reported. Based on this distinction, we propose a process model matrix for AI forensics across four phases: collection, preservation, analysis, and reporting. We also introduce an order of volatility for AI systems, covering runtime state, context windows, logs, retrieval stores, model artifacts, and training lineage. From this matrix, we derive an access conditioned examination framework and identify open research problems, including black box preservation, model version attestation, uncertainty quantification for surrogate based analysis, and chain of custody for mutable AI artifacts.