# Towards a consistent interpretation of AIOps models

arXiv: https://arxiv.org/abs/2202.02298 · HTML: https://mlsanity.com/paper/2202.02298/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 98 references that are in the map.
- First posted on arXiv 4 Feb 2022.

## Builds on (most cited first, 17 of 17)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)](https://mlsanity.com/paper/2003.12206/) (2020)
- [On the (In)fidelity and Sensitivity for Explanations](https://mlsanity.com/paper/1901.09392/) (2019)
- [The Impact of Automated Parameter Optimization on Defect Prediction Models](https://mlsanity.com/paper/1801.10270/) (2018)
- [Evaluating Explanation Methods for Deep Learning in Security](https://mlsanity.com/paper/1906.02108/) (2019)
- [Can I Trust the Explainer? Verifying Post-hoc Explanatory Methods](https://mlsanity.com/paper/1910.02065/) (2019)
- [On the "Naturalness" of Buggy Code](https://mlsanity.com/paper/1506.01159/) (2015)
- [The Impact of Class Rebalancing Techniques on the Performance and Interpretation of Defect Prediction Models](https://mlsanity.com/paper/1801.10269/) (2018)
- [Can We Trust Your Explanations? Sanity Checks for Interpreters in Android Malware Analysis](https://mlsanity.com/paper/2008.05895/) (2020)
- [DeCaf: Diagnosing and Triaging Performance Issues in Large-Scale Cloud Services](https://mlsanity.com/paper/1910.05339/) (2019)
- [Bellwethers: A Baseline Method For Transfer Learning](https://mlsanity.com/paper/1703.06218/) (2017)
- [Run, Forest, Run? On Randomization and Reproducibility in Predictive Software Engineering](https://mlsanity.com/paper/2012.08387/) (2020)
- [SLA Violation Prediction In Cloud Computing: A Machine Learning Perspective](https://mlsanity.com/paper/1611.10338/) (2016)

## Built on by (most cited first, 8 of 8)

- [A Closer Look at AUROC and AUPRC under Class Imbalance](https://mlsanity.com/paper/2401.06091/) (2024)
- [Is Your Anomaly Detector Ready for Change? Adapting AIOps Solutions to the Real World](https://mlsanity.com/paper/2311.10421/) (2023)
- [On-Premise AIOps Infrastructure for a Software Editor SME: An Experience Report](https://mlsanity.com/paper/2308.11225/) (2023)
- [McUDI: Model-Centric Unsupervised Degradation Indicator for Failure Prediction AIOps Solutions](https://mlsanity.com/paper/2401.14093/) (2024)
- [An Efficient Model Maintenance Approach for MLOps](https://mlsanity.com/paper/2412.04657/) (2024)
- [Sustainable Machine Learning Retraining: Optimizing Energy Efficiency Without Compromising Accuracy](https://mlsanity.com/paper/2506.13838/) (2025)
- [Prepared for the Unknown: Adapting AIOps Capacity Forecasting Models to Data Changes](https://mlsanity.com/paper/2510.10320/) (2025)
- [Perturb Your Data: Paraphrase-Guided Training Data Watermarking](https://mlsanity.com/paper/2512.17075/) (2025)

## Abstract (arXiv)

Artificial Intelligence for IT Operations (AIOps) has been adopted in organizations in various tasks, including interpreting models to identify indicators of service failures. To avoid misleading practitioners, AIOps model interpretations should be consistent (i.e., different AIOps models on the same task agree with one another on feature importance). However, many AIOps studies violate established practices in the machine learning community when deriving interpretations, such as interpreting models with suboptimal performance, though the impact of such violations on the interpretation consistency has not been studied. In this paper, we investigate the consistency of AIOps model interpretation along three dimensions: internal consistency, external consistency, and time consistency. We conduct a case study on two AIOps tasks: predicting Google cluster job failures, and Backblaze hard drive failures. We find that the randomness from learners, hyperparameter tuning, and data sampling should be controlled to generate consistent interpretations. AIOps models with AUCs greater than 0.75 yield more consistent interpretation compared to low-performing models. Finally, AIOps models that are constructed with the Sliding Window or Full History approaches have the most consistent interpretation with the trends presented in the entire datasets. Our study provides valuable guidelines for practitioners to derive consistent AIOps model interpretation.