# A Comprehensive Survey of Logging in Software: From Logging Statements Automation to Log Mining and Analysis

arXiv: https://arxiv.org/abs/2110.12489 · HTML: https://mlsanity.com/paper/2110.12489/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 247 references that are in the map.
- First posted on arXiv 24 Oct 2021.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Log-based Anomaly Detection Without Log Parsing](https://mlsanity.com/paper/2108.01955/) (2021)
- [Tools and Benchmarks for Automated Log Parsing](https://mlsanity.com/paper/1811.03509/) (2018)
- [A Survey on Automated Log Analysis for Reliability Engineering](https://mlsanity.com/paper/2009.07237/) (2020)
- [Oreo: Detection of Clones in the Twilight Zone](https://mlsanity.com/paper/1806.05837/) (2018)
- [Logzip: Extracting Hidden Structures via Iterative Clustering for Log Compression](https://mlsanity.com/paper/1910.00409/) (2019)
- [Detection of Early-Stage Enterprise Infection by Mining Large-Scale Log Data](https://mlsanity.com/paper/1411.5005/) (2014)
- [How Bad Can a Bug Get? An Empirical Analysis of Software Failures in the OpenStack Cloud Computing Platform](https://mlsanity.com/paper/1907.04055/) (2019)
- [Tiresias: Predicting Security Events Through Deep Learning](https://mlsanity.com/paper/1905.10328/) (2019)
- [Logram: Efficient Log Parsing Using n-Gram Dictionaries](https://mlsanity.com/paper/2001.03038/) (2020)
- [Studying Duplicate Logging Statements and Their Relationships with Code Clones](https://mlsanity.com/paper/2106.00339/) (2021)
- [Extracting Traffic Primitives Directly from Naturalistically Logged Data for Self-Driving Applications](https://mlsanity.com/paper/1709.03553/) (2017)
- [An Exploratory Study of Log Placement Recommendation in an Enterprise System](https://mlsanity.com/paper/2103.01755/) (2021)
- [What Distributed Systems Say: A Study of Seven Spark Application Logs](https://mlsanity.com/paper/2108.08395/) (2021)
- [Log-based software monitoring: a systematic mapping study](https://mlsanity.com/paper/1912.05878/) (2019)
- [Effective Removal of Operational Log Messages: an Application to Model Inference](https://mlsanity.com/paper/2004.07194/) (2020)

## Built on by (most cited first, 6 of 6)

- [Task-Aware Reduction for Scalable LLM-Database Systems](https://mlsanity.com/paper/2510.11813/) (2025)
- [Benchmarking Small Language Models and Small Reasoning Language Models on System Log Severity Classification](https://mlsanity.com/paper/2601.07790/) (2026)
- [LogSieve: Task-Aware CI Log Reduction for Sustainable LLM-Based Analysis](https://mlsanity.com/paper/2601.20148/) (2026)
- [AIOps-Driven Enhancement of Log Anomaly Detection in Unsupervised Scenarios](https://mlsanity.com/paper/2311.02621/) (2023)
- [TraceSynth: Generating Production-Quality Kernel Traces with Constraint-Guided Diffusion Models](https://mlsanity.com/paper/2607.12104/) (2026)
- [Exploring Block Anomaly Detection In HDFS Log Data Analysis](https://mlsanity.com/paper/2607.29383/) (2026)

## Abstract (arXiv)

Logs are widely used to record runtime information of software systems, such as the timestamp and the importance of an event, the unique ID of the source of the log, and a part of the state of a task's execution. The rich information of logs enables system developers (and operators) to monitor the runtime behaviors of their systems and further track down system problems and perform analysis on log data in production settings. However, the prior research on utilizing logs is scattered and that limits the ability of new researchers in this field to quickly get to the speed and hampers currently active researchers to advance this field further. Therefore, this paper surveys and provides a systematic literature review and mapping of the contemporary logging practices and log statements' mining and monitoring techniques and their applications such as in system failure detection and diagnosis. We study a large number of conference and journal papers that appeared on top-level peer-reviewed venues. Additionally, we draw high-level trends of ongoing research and categorize publications into subdivisions. In the end, and based on our holistic observations during this survey, we provide a set of challenges and opportunities that will lead the researchers in academia and industry in moving the field forward.