# PIDSMaker: Building and Evaluating Provenance-based Intrusion Detection Systems

arXiv: https://arxiv.org/abs/2601.22983 · HTML: https://mlsanity.com/paper/2601.22983/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 82 references that are in the map.
- First posted on arXiv 30 Jan 2026.

## Builds on (most cited first, 26 of 26)

- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- [Temporal Graph Networks for Deep Learning on Dynamic Graphs](https://mlsanity.com/paper/2006.10637/) (2020)
- [Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation](https://mlsanity.com/paper/2010.16061/) (2020)
- [UNICORN: Runtime Provenance-Based Detector for Advanced Persistent Threats](https://mlsanity.com/paper/2001.01525/) (2020)
- [Nondeterminism and Instability in Neural Network Optimization](https://mlsanity.com/paper/2103.04514/) (2021)
- [HOLMES: Real-time APT Detection through Correlation of Suspicious Information Flows](https://mlsanity.com/paper/1810.01594/) (2018)
- [Fast Memory-efficient Anomaly Detection in Streaming Heterogeneous Graphs](https://mlsanity.com/paper/1602.04844/) (2016)
- [Kairos: Practical Intrusion Detection and Investigation using Whole-system Provenance](https://mlsanity.com/paper/2308.05034/) (2023)
- [On the Reproducibility of Neural Network Predictions](https://mlsanity.com/paper/2102.03349/) (2021)
- [SLEUTH: Real-time Attack Scenario Reconstruction from COTS Audit Data](https://mlsanity.com/paper/1801.02062/) (2018)
- [Dynamic Malware Analysis with Feature Engineering and Feature Learning](https://mlsanity.com/paper/1907.07352/) (2019)
- [threaTrace: Detecting and Tracing Host-based Threats in Node Level Through Provenance Graph Learning](https://mlsanity.com/paper/2111.04333/) (2021)
- [MAGIC: Detecting Advanced Persistent Threats via Masked Graph Representation Learning](https://mlsanity.com/paper/2310.09831/) (2023)
- [Practical Whole-System Provenance Capture](https://mlsanity.com/paper/1711.05296/) (2017)
- [NODLINK: An Online System for Fine-Grained APT Attack Detection and Investigation](https://mlsanity.com/paper/2311.02331/) (2023)
- [SIGL: Securing Software Installations Through Deep Graph Learning](https://mlsanity.com/paper/2008.11533/) (2020)
- [Are we there yet? An Industrial Viewpoint on Provenance-based Endpoint Detection and Response Tools](https://mlsanity.com/paper/2307.08349/) (2023)
- [ATLASv2: ATLAS Attack Engagements, Version 2](https://mlsanity.com/paper/2401.01341/) (2023)
- [FRAPpuccino: Fault-detection through Runtime Analysis of Provenance](https://mlsanity.com/paper/1711.11487/) (2017)
- [Incorporating Gradients to Rules: Towards Lightweight, Adaptive Provenance-based Intrusion Detection](https://mlsanity.com/paper/2404.14720/) (2024)
- [Xanthus: Push-button Orchestration of Host Provenance Data Collection](https://mlsanity.com/paper/2005.04717/) (2020)
- [ProvMark: A Provenance Expressiveness Benchmarking System](https://mlsanity.com/paper/1909.11187/) (2019)

## Built on by (most cited first, 1 of 1)

- [HIDBench: Benchmarking Large Language Models for Host-Based Intrusion Detection](https://mlsanity.com/paper/2605.21773/) (2026)

## Abstract (arXiv)

Recent provenance-based intrusion detection systems (PIDSs) have demonstrated strong potential for detecting advanced persistent threats (APTs) by applying machine learning to system provenance graphs. However, evaluating and comparing PIDSs remains difficult: prior work uses inconsistent preprocessing pipelines, non-standard dataset splits, and incompatible ground-truth labeling and metrics. These discrepancies undermine reproducibility, impede fair comparison, and impose substantial re-implementation overhead on researchers. We present PIDSMaker, an open-source framework for developing and evaluating PIDSs under consistent protocols. PIDSMaker consolidates eight state-of-the-art systems into a modular, extensible architecture with standardized preprocessing and ground-truth labels, enabling consistent experiments and apples-to-apples comparisons. A YAML-based configuration interface supports rapid prototyping by composing components across systems without code changes. PIDSMaker also includes utilities for ablation studies, hyperparameter tuning, multi-run instability measurement, and visualization, addressing methodological gaps identified in prior work. We demonstrate PIDSMaker through concrete use cases and release it with preprocessed datasets and labels to support shared evaluation for the PIDS community.