# Reducing Events to Augment Log-based Anomaly Detection Models: An Empirical Study

arXiv: https://arxiv.org/abs/2409.04834 · HTML: https://mlsanity.com/paper/2409.04834/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 56 references that are in the map.
- First posted on arXiv 7 Sep 2024.

## Builds on (most cited first, 8 of 8)

- [LogBERT: Log Anomaly Detection via BERT](https://mlsanity.com/paper/2103.04475/) (2021)
- [Extended Isolation Forest](https://mlsanity.com/paper/1811.02141/) (2018)
- [Log-based Anomaly Detection with Deep Learning: How Far Are We?](https://mlsanity.com/paper/2202.04301/) (2022)
- [Deep Learning for Anomaly Detection in Log Data: A Survey](https://mlsanity.com/paper/2207.03820/) (2022)
- [UniParser: A Unified Log Parser for Heterogeneous Log Data](https://mlsanity.com/paper/2202.06569/) (2022)
- [Self-Supervised Log Parsing](https://mlsanity.com/paper/2003.07905/) (2020)
- [Logzip: Extracting Hidden Structures via Iterative Clustering for Log Compression](https://mlsanity.com/paper/1910.00409/) (2019)
- [Multivariate Log-based Anomaly Detection for Distributed Database](https://mlsanity.com/paper/2406.07976/) (2024)

## Built on by (most cited first, 8 of 8)

- [AgentFM: Role-Aware Failure Management for Distributed Databases with LLM-Driven Multi-Agents](https://mlsanity.com/paper/2504.06614/) (2025)
- [A Survey of AIOps in the Era of Large Language Models](https://mlsanity.com/paper/2507.12472/) (2025)
- [Adaptive Root Cause Localization for Microservice Systems with Multi-Agent Recursion-of-Thought](https://mlsanity.com/paper/2508.20370/) (2025)
- [Agentic Memory Enhanced Recursive Reasoning for Root Cause Localization in Microservices](https://mlsanity.com/paper/2601.02732/) (2026)
- [Hypothesize-Then-Verify: Speculative Root Cause Analysis for Microservices with Pathwise Parallelism](https://mlsanity.com/paper/2601.02736/) (2026)
- [From Feedback Loops to Policy Updates: Reinforcement Fine-Tuning for LLM-Based Alpha Factor Discovery](https://mlsanity.com/paper/2605.15412/) (2026)
- [Towards Robust LLM Post-Training: Automatic Failure Management for Reinforcement Fine-Tuning](https://mlsanity.com/paper/2605.04431/) (2026)
- [Playing social deduction games with reinforcement fine-tuned large language models](https://mlsanity.com/paper/2610.04261/) (2026)

## Abstract (arXiv)

As software systems grow increasingly intricate, the precise detection of anomalies have become both essential and challenging. Current log-based anomaly detection methods depend heavily on vast amounts of log data leading to inefficient inference and potential misguidance by noise logs. However, the quantitative effects of log reduction on the effectiveness of anomaly detection remain unexplored. Therefore, we first conduct a comprehensive study on six distinct models spanning three datasets. Through the study, the impact of log quantity and their effectiveness in representing anomalies is qualifies, uncovering three distinctive log event types that differently influence model performance. Drawing from these insights, we propose LogCleaner: an efficient methodology for the automatic reduction of log events in the context of anomaly detection. Serving as middleware between software systems and models, LogCleaner continuously updates and filters anti-events and duplicative-events in the raw generated logs. Experimental outcomes highlight LogCleaner's capability to reduce over 70% of log events in anomaly detection, accelerating the model's inference speed by approximately 300%, and universally improving the performance of models for anomaly detection.