AI Forensics Across White-, Grey-, and Black-Box Access: A Process Model and Research Agenda for Post-Incident Investigation of AI Systems
Not on the map. Why?
Abstract (arXiv)
AI systems are increasingly involved in decisions and actions that may later require investigation. When an AI related incident occurs, investigators need to reconstruct what the system did, why it behaved that way, and which part of the system or supply chain contributed to the outcome. Existing work on AI forensics remains fragmented, often focusing on a specific system type, artifact, or analysis technique. This paper argues that investigator access is a useful starting point for organizing the field. We distinguish white box, grey box, and black box access and show how each access level changes what can be collected, preserved, analyzed, and reported. Based on this distinction, we propose a process model matrix for AI forensics across four phases: collection, preservation, analysis, and reporting. We also introduce an order of volatility for AI systems, covering runtime state, context windows, logs, retrieval stores, model artifacts, and training lineage. From this matrix, we derive an access conditioned examination framework and identify open research problems, including black box preservation, model version attestation, uncertainty quantification for surrogate based analysis, and chain of custody for mutable AI artifacts.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 8 of its 24 references that are in the map. First posted on arXiv 4 Aug 2026.
Builds on
8 of 24 references.
References · 8
- 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
- 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,543
- 2016Stealing Machine Learning Models via Prediction APIscited by 602
- 2021Towards Observability for Production Machine Learning Pipelinescited by 10
- 2023A Comprehensive Analysis of the Role of Artificial Intelligence and Machine Learning in Modern Digital Forensics and Incident Responsecited by 8
- 2019Founding The Domain of AI Forensicscited by 4
- 2026Foundations for Agentic AI Investigations from the Forensic Analysis of OpenClawcited by 2
- 2020Towards AI Forensics: Did the Artificial Intelligence System Do It?cited by 1
Built on by
Not cited yet.