Challenges and Solutions to Build a Data Pipeline to Identify Anomalies in Enterprise System Performance
Not on the map. Why?
Abstract (arXiv)
We discuss how VMware is solving the following challenges to harness data to operate our ML-based anomaly detection system to detect performance issues in our Software Defined Data Center (SDDC) enterprise deployments: (i) label scarcity and label bias due to heavy dependency on unscalable human annotators, and (ii) data drifts due to ever-changing workload patterns, software stack and underlying hardware. Our anomaly detection system has been deployed in production for many years and has successfully detected numerous major performance issues. We demonstrate that by addressing these data challenges, we not only improve the accuracy of our performance anomaly detection model by 30%, but also ensure that the model performance to never degrade over time.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 6 of its 14 references that are in the map. First posted on arXiv 13 Dec 2021.
Builds on
6 of 14 references.
References · 6
- 2016Equality of Opportunity in Supervised Learningcited by 2,000
- 2020Learning under Concept Drift: A Reviewcited by 470
- 2019Confident Learning: Estimating Uncertainty in Dataset Labelscited by 411
- 2019Identifying and Correcting Label Bias in Machine Learningcited by 146
- 2020Detection of data drift and outliers affecting machine learning model performance over timecited by 19
- 2020Sequential Drift Detection in Deep Learning Classifierscited by 2
Built on by
Not cited yet.