# ADALog: Adaptive Unsupervised Anomaly detection in Logs with Self-attention Masked Language Model

arXiv: https://arxiv.org/abs/2505.13496 · HTML: https://mlsanity.com/paper/2505.13496/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 32 references that are in the map.
- First posted on arXiv 15 May 2025.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little](https://mlsanity.com/paper/2104.06644/) (2021)
- [LogBERT: Log Anomaly Detection via BERT](https://mlsanity.com/paper/2103.04475/) (2021)
- [Log-based Anomaly Detection with Deep Learning: How Far Are We?](https://mlsanity.com/paper/2202.04301/) (2022)
- [Self-Attentive Classification-Based Anomaly Detection in Unstructured Logs](https://mlsanity.com/paper/2008.09340/) (2020)
- [LAnoBERT: System Log Anomaly Detection based on BERT Masked Language Model](https://mlsanity.com/paper/2111.09564/) (2021)
- [Deep Learning for Anomaly Detection in Log Data: A Survey](https://mlsanity.com/paper/2207.03820/) (2022)
- [RAGLog: Log Anomaly Detection using Retrieval Augmented Generation](https://mlsanity.com/paper/2311.05261/) (2023)
- [LogFormer: A Pre-train and Tuning Pipeline for Log Anomaly Detection](https://mlsanity.com/paper/2401.04749/) (2024)
- [TransLog: A Unified Transformer-based Framework for Log Anomaly Detection](https://mlsanity.com/paper/2201.00016/) (2021)
- [On the Effectiveness of Log Representation for Log-based Anomaly Detection](https://mlsanity.com/paper/2308.08736/) (2023)
- [Impact of Log Parsing on Deep Learning-Based Anomaly Detection](https://mlsanity.com/paper/2305.15897/) (2023)
- [A Comprehensive Study of Machine Learning Techniques for Log-Based Anomaly Detection](https://mlsanity.com/paper/2307.16714/) (2023)
- [LogELECTRA: Self-supervised Anomaly Detection for Unstructured Logs](https://mlsanity.com/paper/2402.10397/) (2024)
- [Log2graphs: An Unsupervised Framework for Log Anomaly Detection with Efficient Feature Extraction](https://mlsanity.com/paper/2409.11890/) (2024)
- [Log-based Anomaly Detection of Enterprise Software: An Empirical Study](https://mlsanity.com/paper/2310.20492/) (2023)

## Built on by (most cited first, 2 of 2)

- [LogbQuant: Quantizing Language Models in Logarithmic Space](https://mlsanity.com/paper/2607.01127/) (2026)
- [X-WAD: eXplainable Web Anomaly Detection](https://mlsanity.com/paper/2608.27172/) (2026)

## Abstract (arXiv)

Modern software systems generate extensive heterogeneous log data with dynamic formats, fragmented event sequences, and varying temporal patterns, making anomaly detection both crucial and challenging. To address these complexities, we propose ADALog, an adaptive, unsupervised anomaly detection framework designed for practical applicability across diverse real-world environments. Unlike traditional methods reliant on log parsing, strict sequence dependencies, or labeled data, ADALog operates on individual unstructured logs, extracts intra-log contextual relationships, and performs adaptive thresholding on normal data. The proposed approach utilizes a transformer-based, pretrained bidirectional encoder with a masked language modeling task, fine-tuned on normal logs to capture domain-specific syntactic and semantic patterns essential for accurate anomaly detection. Anomalies are identified via token-level reconstruction probabilities, aggregated into log-level scores, with adaptive percentile-based thresholding calibrated only on normal data. This allows the model to dynamically adapt to evolving system behaviors while avoiding rigid, heuristic-based thresholds common in traditional systems. We evaluate ADALog on benchmark datasets BGL, Thunderbird, and Spirit, showing strong generalization and competitive performance compared to state-of-the-art supervised and unsupervised methods. Additional ablation studies examine the effects of masking, fine-tuning, and token positioning on model behavior and interpretability.