# Monitoring and explainability of models in production

arXiv: https://arxiv.org/abs/2007.06299 · HTML: https://mlsanity.com/paper/2007.06299/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jul 2020.

## Built on by (most cited first, 24 of 24)

- [MLOps -- Definitions, Tools and Challenges](https://mlsanity.com/paper/2201.00162/) (2022)
- [Ergo, SMIRK is Safe: A Safety Case for a Machine Learning Component in a Pedestrian Automatic Emergency Brake System](https://mlsanity.com/paper/2204.07874/) (2022)
- [ModelCI-e: Enabling Continual Learning in Deep Learning Serving Systems](https://mlsanity.com/paper/2106.03122/) (2021)
- [Context-Aware Drift Detection](https://mlsanity.com/paper/2203.08644/) (2022)
- [Estimating Model Performance Under Covariate Shift Without Labels](https://mlsanity.com/paper/2401.08348/) (2024)
- [Confidence-based Estimators for Predictive Performance in Model Monitoring](https://mlsanity.com/paper/2407.08649/) (2024)
- [CheXstray: Real-time Multi-Modal Data Concordance for Drift Detection in Medical Imaging AI](https://mlsanity.com/paper/2202.02833/) (2022)
- [Explanation Shift: How Did the Distribution Shift Impact the Model?](https://mlsanity.com/paper/2303.08081/) (2023)
- [Desiderata for next generation of ML model serving](https://mlsanity.com/paper/2210.14665/) (2022)
- [A monitoring framework for deployed machine learning models with supply chain examples](https://mlsanity.com/paper/2211.06239/) (2022)
- [DEPLOYR: A technical framework for deploying custom real-time machine learning models into the electronic medical record](https://mlsanity.com/paper/2303.06269/) (2023)
- [Pitfalls of Explainable ML: An Industry Perspective](https://mlsanity.com/paper/2106.07758/) (2021)
- [Towards Trustworthy Machine Learning in Production: An Overview of the Robustness in MLOps Approach](https://mlsanity.com/paper/2410.21346/) (2024)
- [Monitoring Machine Learning Models: Online Detection of Relevant Deviations](https://mlsanity.com/paper/2309.15187/) (2023)
- [Designing monitoring strategies for deployed machine learning algorithms: navigating performativity through a causal lens](https://mlsanity.com/paper/2311.11463/) (2023)
- [Ensembling Shift Detectors: an Extensive Empirical Evaluation](https://mlsanity.com/paper/2106.14608/) (2021)
- [Causal fault localisation in dataflow systems](https://mlsanity.com/paper/2304.11987/) (2023)
- [Machine Learning Operations: A Mapping Study](https://mlsanity.com/paper/2409.19416/) (2024)
- [Computer vision based vehicle tracking as a complementary and scalable approach to RFID tagging](https://mlsanity.com/paper/2209.05911/) (2022)
- [Perspectives on a Reliability Monitoring Framework for Agentic AI Systems](https://mlsanity.com/paper/2511.09178/) (2025)
- [Latent space analysis and generalization to out-of-distribution data](https://mlsanity.com/paper/2511.15010/) (2025)
- [ShapShift: Explaining Model Prediction Shifts with Subgroup Conditional Shapley Values](https://mlsanity.com/paper/2604.11200/) (2026)
- [Tabular Deep Learning for Algorithmic Trading: Cross-Regime Bayesian Optimisation for Equity Signal Generation](https://mlsanity.com/paper/2608.27076/) (2026)
- [TRACE: A Governance Framework for Measuring Explainability Debt in Production AI Systems](https://mlsanity.com/paper/2610.10957/) (2026)

## Abstract (arXiv)

The machine learning lifecycle extends beyond the deployment stage. Monitoring deployed models is crucial for continued provision of high quality machine learning enabled services. Key areas include model performance and data monitoring, detecting outliers and data drift using statistical techniques, and providing explanations of historic predictions. We discuss the challenges to successful implementation of solutions in each of these areas with some recent examples of production ready solutions using open source tools.