# Scalable Temporal Anomaly Causality Discovery in Large Systems: Achieving Computational Efficiency with Binary Anomaly Flag Data

arXiv: https://arxiv.org/abs/2412.11800 · HTML: https://mlsanity.com/paper/2412.11800/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 97 references that are in the map.
- First posted on arXiv 16 Dec 2024.

## Builds on (most cited first, 30 of 30)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Detecting causal associations in large nonlinear time series datasets](https://mlsanity.com/paper/1702.07007/) (2017)
- [Gradient-Based Neural DAG Learning](https://mlsanity.com/paper/1906.02226/) (2019)
- [On the Role of Sparsity and DAG Constraints for Learning Linear DAGs](https://mlsanity.com/paper/2006.10201/) (2020)
- [CAM: Causal additive models, high-dimensional order search and penalized regression](https://mlsanity.com/paper/1310.1533/) (2013)
- [Time-Series Anomaly Detection Service at Microsoft](https://mlsanity.com/paper/1906.03821/) (2019)
- [Causal Discovery with Reinforcement Learning](https://mlsanity.com/paper/1906.04477/) (2019)
- [Discovering contemporaneous and lagged causal relations in autocorrelated nonlinear time series datasets](https://mlsanity.com/paper/2003.03685/) (2020)
- [Multivariate LSTM-FCNs for Time Series Classification](https://mlsanity.com/paper/1801.04503/) (2018)
- [High-recall causal discovery for autocorrelated time series with latent confounders](https://mlsanity.com/paper/2007.01884/) (2020)
- [Causal structure based root cause analysis of outliers](https://mlsanity.com/paper/1912.02724/) (2019)
- [Granger Causality: A Review and Recent Advances](https://mlsanity.com/paper/2105.02675/) (2021)
- [gCastle: A Python Toolbox for Causal Discovery](https://mlsanity.com/paper/2111.15155/) (2021)
- [A Graph Autoencoder Approach to Causal Structure Learning](https://mlsanity.com/paper/1911.07420/) (2019)
- [Ordering-Based Causal Discovery with Reinforcement Learning](https://mlsanity.com/paper/2105.06631/) (2021)
- [Why did the distribution change?](https://mlsanity.com/paper/2102.13384/) (2021)
- [Root Cause Identification for Collective Anomalies in Time Series given an Acyclic Summary Causal Graph with Loops](https://mlsanity.com/paper/2303.04038/) (2023)
- [CausalBench: A Large-scale Benchmark for Network Inference from Single-cell Perturbation Data](https://mlsanity.com/paper/2210.17283/) (2022)
- [Reconstructing regime-dependent causal relationships from observational time series](https://mlsanity.com/paper/2007.00267/) (2020)
- [ClaSP -- Parameter-free Time Series Segmentation](https://mlsanity.com/paper/2207.13987/) (2022)
- [Enhancing Causal Discovery from Robot Sensor Data in Dynamic Scenarios](https://mlsanity.com/paper/2302.10135/) (2023)
- [Comprehensive Review and Empirical Evaluation of Causal Discovery Algorithms for Numerical Data](https://mlsanity.com/paper/2407.13054/) (2024)
- [Root-cause Analysis for Time-series Anomalies via Spatiotemporal Graphical Modeling in Distributed Complex Systems](https://mlsanity.com/paper/1805.12296/) (2018)
- [Spatio-Temporal Anomaly Detection with Graph Networks for Data Quality Monitoring of the Hadron Calorimeter](https://mlsanity.com/paper/2311.04190/) (2023)
- [Statistical Perspective on Functional and Causal Neural Connectomics: The Time-Aware PC Algorithm](https://mlsanity.com/paper/2204.04845/) (2022)
- [The model of an anomaly detector for HiLumi LHC magnets based on Recurrent Neural Networks and adaptive quantization](https://mlsanity.com/paper/1709.09883/) (2017)
- [Data Quality Monitoring for the Hadron Calorimeters Using Transfer Learning for Anomaly Detection](https://mlsanity.com/paper/2408.16612/) (2024)
- [The CMS monitoring infrastructure and applications](https://mlsanity.com/paper/2007.03630/) (2020)
- [Framework for inferring empirical causal graphs from binary data to support multidimensional poverty analysis](https://mlsanity.com/paper/2205.06131/) (2022)
- [Lightweight Multi-System Multivariate Interconnection and Divergence Discovery](https://mlsanity.com/paper/2404.08453/) (2024)

## Abstract (arXiv)

Extracting anomaly causality facilitates diagnostics once monitoring systems detect system faults. Identifying anomaly causes in large systems involves investigating a broader set of monitoring variables across multiple subsystems. However, learning graphical causal models (GCMs) comes with a significant computational burden that restrains the applicability of most existing methods in real-time and large-scale deployments. In addition, modern monitoring applications for large systems often generate large amounts of binary alarm flags, and the distinct characteristics of binary anomaly data -- the meaning of state transition and data sparsity -- challenge existing causality learning mechanisms. This study proposes an anomaly causal discovery approach (AnomalyCD), addressing the accuracy and computational challenges of generating GCMs from temporal binary flag datasets. The AnomalyCD presents several strategies, such as anomaly data-aware causality testing, sparse data and prior link compression, and edge pruning adjustment approaches. We validate the performance of the approach on two datasets: monitoring sensor data from the readout-box system of the Compact Muon Solenoid experiment at CERN, and a public dataset from an information technology monitoring system. The results on temporal GCMs demonstrate a considerable reduction of computation overhead and a moderate enhancement of accuracy on the binary anomaly datasets. Code: https://github.com/muleina/AnomalyCD .