# Surrogate Modeling and Explainable Artificial Intelligence for Complex Systems: A Workflow for Automated Simulation Exploration

arXiv: https://arxiv.org/abs/2510.16742 · HTML: https://mlsanity.com/paper/2510.16742/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 106 references that are in the map.
- First posted on arXiv 19 Oct 2025.

## Builds on (most cited first, 20 of 20)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Models](https://mlsanity.com/paper/1612.08468/) (2016)
- [SMT 2.0: A Surrogate Modeling Toolbox with a focus on Hierarchical and Mixed Variables Gaussian Processes](https://mlsanity.com/paper/2305.13998/) (2023)
- [SHAFF: Fast and consistent SHApley eFfect estimates via random Forests](https://mlsanity.com/paper/2105.11724/) (2021)
- [A mixed-categorical correlation kernel for Gaussian process](https://mlsanity.com/paper/2211.08262/) (2022)
- [Interpretability Needs a New Paradigm](https://mlsanity.com/paper/2405.05386/) (2024)
- [Surrogate Locally-Interpretable Models with Supervised Machine Learning Algorithms](https://mlsanity.com/paper/2007.14528/) (2020)
- [High-dimensional mixed-categorical Gaussian processes with application to multidisciplinary design optimization for a green aircraft](https://mlsanity.com/paper/2311.06130/) (2023)
- [Surrogate Modeling and Control of Medical Digital Twins](https://mlsanity.com/paper/2402.05750/) (2024)
- [System Architecture Optimization Strategies: Dealing with Expensive Hierarchical Problems](https://mlsanity.com/paper/2502.00838/) (2025)
- [Investigating the Duality of Interpretability and Explainability in Machine Learning](https://mlsanity.com/paper/2503.21356/) (2025)
- [Which LIME should I trust? Concepts, Challenges, and Solutions](https://mlsanity.com/paper/2503.24365/) (2025)
- [Variance based sensitivity analysis for Monte Carlo and importance sampling reliability assessment with Gaussian processes](https://mlsanity.com/paper/2011.15001/) (2020)
- [Parameter selection in Gaussian process interpolation: an empirical study of selection criteria](https://mlsanity.com/paper/2107.06006/) (2021)
- [A Surrogate Model Framework for Explainable Autonomous Behaviour](https://mlsanity.com/paper/2305.19724/) (2023)
- [SMT-EX: An Explainable Surrogate Modeling Toolbox for Mixed-Variables Design Exploration](https://mlsanity.com/paper/2503.19496/) (2025)
- [Bayesian Optimization of a Lightweight and Accurate Neural Network for Aerodynamic Performance Prediction](https://mlsanity.com/paper/2503.19479/) (2025)
- [Surrogate-based optimization of system architectures subject to hidden constraints](https://mlsanity.com/paper/2504.08721/) (2025)
- [Beyond Shapley Values: Cooperative Games for the Interpretation of Machine Learning Models](https://mlsanity.com/paper/2506.13900/) (2025)
- [Hacking a surrogate model approach to XAI](https://mlsanity.com/paper/2406.16626/) (2024)

## Built on by (most cited first, 6 of 6)

- [Global Sensitivity Analysis for Engineering Design Based on Individual Conditional Expectations](https://mlsanity.com/paper/2512.11946/) (2025)
- [Sustainable Materials Discovery in the Era of Artificial Intelligence](https://mlsanity.com/paper/2601.21527/) (2026)
- [Analyzing Shapley Additive Explanations to Understand Anomaly Detection Algorithm Behaviors and Their Complementarity](https://mlsanity.com/paper/2602.00208/) (2026)
- [From Model-Based Screening to Data-Driven Surrogates: A Multi-Stage Workflow for Exploring Stochastic Agent-Based Models](https://mlsanity.com/paper/2604.03350/) (2026)
- [Estimating Inconsistency Response Surfaces under Uncertainty in Cyber-Physical System Development](https://mlsanity.com/paper/2609.11331/) (2026)
- [Hierarchical Bayesian optimization of an aircraft-based multi-agent system-of-systems](https://mlsanity.com/paper/2609.22130/) (2026)

## Abstract (arXiv)

Complex systems are increasingly explored through simulation-driven engineering workflows that combine physics-based and empirical models with optimization and analytics. Despite their power, these workflows face two central obstacles: (1) high computational cost, since accurate exploration requires many expensive simulator runs; and (2) limited transparency and reliability when decisions rely on opaque blackbox components. We propose a workflow that addresses both challenges by training lightweight emulators on compact designs of experiments that (i) provide fast, low-latency approximations of expensive simulators, (ii) enable rigorous uncertainty quantification, and (iii) are adapted for global and local Explainable Artificial Intelligence (XAI) analyses. This workflow unifies every simulation-based complex-system analysis tool, ranging from engineering design to agent-based models for socio-environmental understanding. In this paper, we proposea comparative methodology and practical recommendations for using surrogate-based explainability tools within the proposed workflow. The methodology supports continuous and categorical inputs, combines global-effect and uncertainty analyses with local attribution, and evaluates the consistency of explanations across surrogate models, thereby diagnosing surrogate adequacy and guiding further data collection or model refinement. We demonstrate the approach on two contrasting case studies: a multidisciplinary design analysis of a hybrid-electric aircraft and an agent-based model of urban segregation. Results show that the surrogate model and XAI coupling enables large-scale exploration in seconds, uncovers nonlinear interactions and emergent behaviors, identifies key design and policy levers, and signals regions where surrogates require more data or alternative architectures.