# What Distributed Systems Say: A Study of Seven Spark Application Logs

arXiv: https://arxiv.org/abs/2108.08395 · HTML: https://mlsanity.com/paper/2108.08395/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Aug 2021.

## Built on by (most cited first, 3 of 3)

- [A Comprehensive Survey of Logging in Software: From Logging Statements Automation to Log Mining and Analysis](https://mlsanity.com/paper/2110.12489/) (2021)
- [Leveraging Code Clones and Natural Language Processing for Log Statement Prediction](https://mlsanity.com/paper/2109.03859/) (2021)
- [Borrowing from Similar Code: A Deep Learning NLP-Based Approach for Log Statement Automation](https://mlsanity.com/paper/2112.01259/) (2021)

## Abstract (arXiv)

Execution logs are a crucial medium as they record runtime information of software systems. Although extensive logs are helpful to provide valuable details to identify the root cause in postmortem analysis in case of a failure, this may also incur performance overhead and storage cost. Therefore, in this research, we present the result of our experimental study on seven Spark benchmarks to illustrate the impact of different logging verbosity levels on the execution time and storage cost of distributed software systems. We also evaluate the log effectiveness and the information gain values, and study the changes in performance and the generated logs for each benchmark with various types of distributed system failures. Our research draws insightful findings for developers and practitioners on how to set up and utilize their distributed systems to benefit from the execution logs.