# Do AI Coding Agents Log Like Humans? An Empirical Study

arXiv: https://arxiv.org/abs/2604.09409 · HTML: https://mlsanity.com/paper/2604.09409/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 17 of its 39 references that are in the map.
- First posted on arXiv 10 Apr 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 17 of 17)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Instruction-Following Evaluation for Large Language Models](https://mlsanity.com/paper/2311.07911/) (2023)
- [LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods](https://mlsanity.com/paper/2412.05579/) (2024)
- [Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models](https://mlsanity.com/paper/2404.18796/) (2024)
- [Lost at C: A User Study on the Security Implications of Large Language Model Code Assistants](https://mlsanity.com/paper/2208.09727/) (2022)
- [A Survey on Large Language Models for Software Engineering](https://mlsanity.com/paper/2312.15223/) (2023)
- [On the Use of Agentic Coding: An Empirical Study of Pull Requests on GitHub](https://mlsanity.com/paper/2509.14745/) (2025)
- [How Do AI Agents Do Human Work? Comparing AI and Human Workflows Across Diverse Occupations](https://mlsanity.com/paper/2510.22780/) (2025)
- [Agentic Refactoring: An Empirical Study of AI Coding Agents](https://mlsanity.com/paper/2511.04824/) (2025)
- [Using Deep Learning to Generate Complete Log Statements](https://mlsanity.com/paper/2201.04837/) (2022)
- [Unveiling ChatGPT's Usage in Open Source Projects: A Mining-based Study](https://mlsanity.com/paper/2402.16480/) (2024)
- [Software Engineering and Foundation Models: Insights from Industry Blogs Using a Jury of Foundation Models](https://mlsanity.com/paper/2410.09012/) (2024)
- [Comparing Human and LLM Generated Code: The Jury is Still Out!](https://mlsanity.com/paper/2501.16857/) (2025)
- [Studying Logging Practice in Machine Learning-based Applications](https://mlsanity.com/paper/2301.04234/) (2023)
- [Learning from Mistakes: Understanding Ad-hoc Logs through Analyzing Accidental Commits](https://mlsanity.com/paper/2501.09892/) (2025)
- [Automated File-Level Logging Generation for Machine Learning Applications using LLMs: A Case Study using GPT-4o Mini](https://mlsanity.com/paper/2508.04820/) (2025)

## Built on by (most cited first, 3 of 3)

- [From Agent Behaviour to Agent-Friendly Documentation: An Empirical Study of How Coding Agents Discover, Read, and Write Technical Documentation](https://mlsanity.com/paper/2608.20195/) (2026)
- [On the Maintenance and Co-evolution of Agent Plugins: An Empirical Study of Claude Code Plugin Marketplaces](https://mlsanity.com/paper/2608.28497/) (2026)
- [Towards Automatically Pruning Logging Code with Coding Agents: How Far Are We?](https://mlsanity.com/paper/2610.04716/) (2026)

## Abstract (arXiv)

Software logging is essential for maintaining and debugging complex systems, yet it remains unclear how AI coding agents handle this non-functional requirement. While prior work characterizes human logging practices, the behaviors of AI coding agents and the efficacy of natural language instructions in governing them are unexplored. To address this gap, we conduct an empirical study of 4,550 agentic pull requests across 81 open-source repositories. We compare agent logging patterns against human baselines and analyze the impact of explicit logging instructions. We find that agents change logging less often than humans in 58.4% of repositories, though they exhibit higher log density when they do. Furthermore, explicit logging instructions are rare (4.7%) and ineffective, as agents fail to comply with constructive requests 67% of the time. Finally, we observe that humans perform 72.5% of post-generation log repairs, acting as "silent janitors" who fix logging and observability issues without explicit review feedback. These findings indicate a dual failure in natural language instruction (i.e., scarcity of logging instructions and low agent compliance), suggesting that deterministic guardrails might be necessary to ensure consistent logging practices.