# Automated File-Level Logging Generation for Machine Learning Applications using LLMs: A Case Study using GPT-4o Mini

arXiv: https://arxiv.org/abs/2508.04820 · HTML: https://mlsanity.com/paper/2508.04820/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 40 references that are in the map.
- First posted on arXiv 6 Aug 2025.

## Builds on (most cited first, 10 of 10)

- [Structured Chain-of-Thought Prompting for Code Generation](https://mlsanity.com/paper/2305.06599/) (2023)
- [Practical Program Repair in the Era of Large Pre-trained Language Models](https://mlsanity.com/paper/2210.14179/) (2022)
- [Verbosity Bias in Preference Labeling by Large Language Models](https://mlsanity.com/paper/2310.10076/) (2023)
- [Using Deep Learning to Generate Complete Log Statements](https://mlsanity.com/paper/2201.04837/) (2022)
- [Exploring the Effectiveness of LLMs in Automated Logging Generation: An Empirical Study](https://mlsanity.com/paper/2307.05950/) (2023)
- [Go Static: Contextualized Logging Statement Generation](https://mlsanity.com/paper/2402.12958/) (2024)
- [FastLog: An End-to-End Method to Efficiently Generate and Insert Logging Statements](https://mlsanity.com/paper/2311.02862/) (2023)
- [AL-Bench: A Benchmark for Automatic Logging](https://mlsanity.com/paper/2502.03160/) (2025)
- [Studying Logging Practice in Machine Learning-based Applications](https://mlsanity.com/paper/2301.04234/) (2023)
- [Log Statements Generation via Deep Learning: Widening the Support Provided to Developers](https://mlsanity.com/paper/2311.04587/) (2023)

## Built on by (most cited first, 1 of 1)

- [Do AI Coding Agents Log Like Humans? An Empirical Study](https://mlsanity.com/paper/2604.09409/) (2026)

## Abstract (arXiv)

Logging is essential in software development, helping developers monitor system behavior and aiding in debugging applications. Given the ability of large language models (LLMs) to generate natural language and code, researchers are exploring their potential to generate log statements. However, prior work focuses on evaluating logs introduced in code functions, leaving file-level log generation underexplored -- especially in machine learning (ML) applications, where comprehensive logging can enhance reliability. In this study, we evaluate the capacity of GPT-4o mini as a case study to generate log statements for ML projects at file level. We gathered a set of 171 ML repositories containing 4,073 Python files with at least one log statement. We identified and removed the original logs from the files, prompted the LLM to generate logs for them, and evaluated both the position of the logs and log level, variables, and text quality of the generated logs compared to human-written logs. In addition, we manually analyzed a representative sample of generated logs to identify common patterns and challenges. We find that the LLM introduces logs in the same place as humans in 63.91% of cases, but at the cost of a high overlogging rate of 82.66%. Furthermore, our manual analysis reveals challenges for file-level logging, which shows overlogging at the beginning or end of a function, difficulty logging within large code blocks, and misalignment with project-specific logging conventions. While the LLM shows promise for generating logs for complete files, these limitations remain to be addressed for practical implementation.