# Scalable and Efficient Large-Scale Log Analysis with LLMs: An IT Software Support Case Study

arXiv: https://arxiv.org/abs/2511.14803 · HTML: https://mlsanity.com/paper/2511.14803/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 35 references that are in the map.
- First posted on arXiv 17 Nov 2025.

## Builds on (most cited first, 9 of 9)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Loghub: A Large Collection of System Log Datasets for AI-driven Log Analytics](https://mlsanity.com/paper/2008.06448/) (2020)
- [Interpretable Online Log Analysis Using Large Language Models with Prompt Strategies](https://mlsanity.com/paper/2308.07610/) (2023)
- [LILAC: Log Parsing using LLMs with Adaptive Parsing Cache](https://mlsanity.com/paper/2310.01796/) (2023)
- [A Large-Scale Evaluation for Log Parsing Techniques: How Far Are We?](https://mlsanity.com/paper/2308.10828/) (2023)
- [LogLM: From Task-based to Instruction-based Automated Log Analysis](https://mlsanity.com/paper/2410.09352/) (2024)
- [Neural Architecture Search for Effective Teacher-Student Knowledge Transfer in Language Models](https://mlsanity.com/paper/2303.09639/) (2023)
- [Learning Representations on Logs for AIOps](https://mlsanity.com/paper/2308.11526/) (2023)
- [Root-KGD: A Novel Framework for Root Cause Diagnosis Based on Knowledge Graph and Industrial Data](https://mlsanity.com/paper/2406.13664/) (2024)

## Abstract (arXiv)

IT environments typically have logging mechanisms to monitor system health and detect issues. However, the huge volume of generated logs makes manual inspection impractical, highlighting the importance of automated log analysis in IT Software Support. In this paper, we propose a log analytics tool that leverages Large Language Models (LLMs) for log data processing and issue diagnosis, enabling the generation of automated insights and summaries. We further present a novel approach for efficiently running LLMs on CPUs to process massive log volumes in minimal time without compromising output quality. We share the insights and lessons learned from deployment of the tool - in production since March 2024 - scaled across 70 software products, processing over 2000 tickets for issue diagnosis, achieving a time savings of 300+ man hours and an estimated 15,444 per month in manpower costs compared to the traditional log analysis practices.