# Logram: Efficient Log Parsing Using n-Gram Dictionaries

arXiv: https://arxiv.org/abs/2001.03038 · HTML: https://mlsanity.com/paper/2001.03038/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jan 2020.

## Built on by (most cited first, 9 of 9)

- [Log-based Anomaly Detection Without Log Parsing](https://mlsanity.com/paper/2108.01955/) (2021)
- [Log-based Anomaly Detection with Deep Learning: How Far Are We?](https://mlsanity.com/paper/2202.04301/) (2022)
- [Experience Report: Deep Learning-based System Log Analysis for Anomaly Detection](https://mlsanity.com/paper/2107.05908/) (2021)
- [LogGD:Detecting Anomalies from System Logs by Graph Neural Networks](https://mlsanity.com/paper/2209.07869/) (2022)
- [A Comprehensive Survey of Logging in Software: From Logging Statements Automation to Log Mining and Analysis](https://mlsanity.com/paper/2110.12489/) (2021)
- [MoniLog: An Automated Log-Based Anomaly Detection System for Cloud Computing Infrastructures](https://mlsanity.com/paper/2304.11940/) (2023)
- [DeLag: Using Multi-Objective Optimization to Enhance the Detection of Latency Degradation Patterns in Service-based Systems](https://mlsanity.com/paper/2110.11155/) (2021)
- [Topic Modeling, Clade-assisted Sentiment Analysis, and Vaccine Brand Reputation Analysis of COVID-19 Vaccine-related Facebook Comments in the Philippines](https://mlsanity.com/paper/2111.04416/) (2021)
- [Interactive Log Parsing via Light-weight User Feedback](https://mlsanity.com/paper/2301.12225/) (2023)

## Abstract (arXiv)

Software systems usually record important runtime information in their logs. Logs help practitioners understand system runtime behaviors and diagnose field failures. As logs are usually very large in size, automated log analysis is needed to assist practitioners in their software operation and maintenance efforts. Typically, the first step of automated log analysis is log parsing, i.e., converting unstructured raw logs into structured data. However, log parsing is challenging, because logs are produced by static templates in the source code (i.e., logging statements) yet the templates are usually inaccessible when parsing logs. Prior work proposed automated log parsing approaches that have achieved high accuracy. However, as the volume of logs grows rapidly in the era of cloud computing, efficiency becomes a major concern in log parsing. In this work, we propose an automated log parsing approach, Logram, which leverages n-gram dictionaries to achieve efficient log parsing. We evaluated Logram on 16 public log datasets and compared Logram with five state-of-the-art log parsing approaches. We found that Logram achieves a similar parsing accuracy to the best existing approaches while outperforms these approaches in efficiency (i.e., 1.8 to 5.1 times faster than the second fastest approaches). Furthermore, we deployed Logram on Spark and we found that Logram scales out efficiently with the number of Spark nodes (e.g., with near-linear scalability) without sacrificing parsing accuracy. In addition, we demonstrated that Logram can support effective online parsing of logs, achieving similar parsing results and efficiency with the offline mode.