# RPG-AE: Neuro-Symbolic Graph Autoencoders with Rare Pattern Mining for Provenance-Based Anomaly Detection

arXiv: https://arxiv.org/abs/2602.02929 · HTML: https://mlsanity.com/paper/2602.02929/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 23 references that are in the map.
- First posted on arXiv 3 Feb 2026.

## Builds on (most cited first, 13 of 13)

- [Variational Graph Auto-Encoders](https://mlsanity.com/paper/1611.07308/) (2016)
- [A Comprehensive Survey on Graph Anomaly Detection with Deep Learning](https://mlsanity.com/paper/2106.07178/) (2021)
- [UNICORN: Runtime Provenance-Based Detector for Advanced Persistent Threats](https://mlsanity.com/paper/2001.01525/) (2020)
- [Fast Memory-efficient Anomaly Detection in Streaming Heterogeneous Graphs](https://mlsanity.com/paper/1602.04844/) (2016)
- [SLEUTH: Real-time Attack Scenario Reconstruction from COTS Audit Data](https://mlsanity.com/paper/1801.02062/) (2018)
- [A baseline for unsupervised advanced persistent threat detection in system-level provenance](https://mlsanity.com/paper/1906.06940/) (2019)
- [Conformal Load Prediction with Transductive Graph Autoencoders](https://mlsanity.com/paper/2406.08281/) (2024)
- [Deep Contrastive One-Class Time Series Anomaly Detection](https://mlsanity.com/paper/2207.01472/) (2022)
- [A Rule Mining-Based Advanced Persistent Threats Detection System](https://mlsanity.com/paper/2105.10053/) (2021)
- [Hack Me If You Can: Aggregating AutoEncoders for Countering Persistent Access Threats Within Highly Imbalanced Data](https://mlsanity.com/paper/2406.19220/) (2024)
- [APT-LLM: Embedding-Based Anomaly Detection of Cyber Advanced Persistent Threats Using Large Language Models](https://mlsanity.com/paper/2502.09385/) (2025)
- [From One Attack Domain to Another: Contrastive Transfer Learning with Siamese Networks for APT Detection](https://mlsanity.com/paper/2511.20500/) (2025)
- [Ranking-Enhanced Anomaly Detection Using Active Learning-Assisted Attention Adversarial Dual AutoEncoders](https://mlsanity.com/paper/2511.20480/) (2025)

## Abstract (arXiv)

Advanced Persistent Threats (APTs) are sophisticated, long-term cyberattacks that are difficult to detect because they operate stealthily and often blend into normal system behavior. This paper presents a neuro-symbolic anomaly detection framework that combines a Graph Autoencoder (GAE) with rare pattern mining to identify APT-like activities in system-level provenance data. Our approach first constructs a process behavioral graph using k-Nearest Neighbors based on feature similarity, then learns normal relational structure using a Graph Autoencoder. Anomaly candidates are identified through deviations between observed and reconstructed graph structure. To further improve detection, we integrate an rare pattern mining module that discovers infrequent behavioral co-occurrences and uses them to boost anomaly scores for processes exhibiting rare signatures. We evaluate the proposed method on the DARPA Transparent Computing datasets and show that rare-pattern boosting yields substantial gains in anomaly ranking quality over the baseline GAE. Compared with existing unsupervised approaches on the same benchmark, our single unified model consistently outperforms individual context-based detectors and achieves performance competitive with ensemble aggregation methods that require multiple separate detectors. These results highlight the value of coupling graph-based representation learning with classical pattern mining to improve both effectiveness and interpretability in provenance-based security anomaly detection.