# ClusterLog: Clustering Logs for Effective Log-based Anomaly Detection

arXiv: https://arxiv.org/abs/2301.07846 · HTML: https://mlsanity.com/paper/2301.07846/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 34 references that are in the map.
- First posted on arXiv 19 Jan 2023.

## Builds on (most cited first, 5 of 5)

- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [LogBERT: Log Anomaly Detection via BERT](https://mlsanity.com/paper/2103.04475/) (2021)
- [Log-based Anomaly Detection Without Log Parsing](https://mlsanity.com/paper/2108.01955/) (2021)
- [Tools and Benchmarks for Automated Log Parsing](https://mlsanity.com/paper/1811.03509/) (2018)
- [Detection of Early-Stage Enterprise Infection by Mining Large-Scale Log Data](https://mlsanity.com/paper/1411.5005/) (2014)

## Abstract (arXiv)

With the increasing prevalence of scalable file systems in the context of High Performance Computing (HPC), the importance of accurate anomaly detection on runtime logs is increasing. But as it currently stands, many state-of-the-art methods for log-based anomaly detection, such as DeepLog, have encountered numerous challenges when applied to logs from many parallel file systems (PFSes), often due to their irregularity and ambiguity in time-based log sequences. To circumvent these problems, this study proposes ClusterLog, a log pre-processing method that clusters the temporal sequence of log keys based on their semantic similarity. By grouping semantically and sentimentally similar logs, this approach aims to represent log sequences with the smallest amount of unique log keys, intending to improve the ability of a downstream sequence-based model to effectively learn the log patterns. The preliminary results of ClusterLog indicate not only its effectiveness in reducing the granularity of log sequences without the loss of important sequence information but also its generalizability to different file systems' logs.