Do AI Coding Agents Log Like Humans? An Empirical Study
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
Software logging is essential for maintaining and debugging complex systems, yet it remains unclear how AI coding agents handle this non-functional requirement. While prior work characterizes human logging practices, the behaviors of AI coding agents and the efficacy of natural language instructions in governing them are unexplored. To address this gap, we conduct an empirical study of 4,550 agentic pull requests across 81 open-source repositories. We compare agent logging patterns against human baselines and analyze the impact of explicit logging instructions. We find that agents change logging less often than humans in 58.4% of repositories, though they exhibit higher log density when they do. Furthermore, explicit logging instructions are rare (4.7%) and ineffective, as agents fail to comply with constructive requests 67% of the time. Finally, we observe that humans perform 72.5% of post-generation log repairs, acting as "silent janitors" who fix logging and observability issues without explicit review feedback. These findings indicate a dual failure in natural language instruction (i.e., scarcity of logging instructions and low agent compliance), suggesting that deterministic guardrails might be necessary to ensure consistent logging practices.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 17 of its 39 references that are in the map. First posted on arXiv 10 Apr 2026.
Builds on
17 of 39 references.
Closest ancestors · 2
Same direction, newest first
- 2025Agentic Refactoring: An Empirical Study of AI Coding Agents6 Nov
- 2025On the Use of Agentic Coding: An Empirical Study of Pull Requests on GitHub18 Sep
Other references · 15
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
- 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
- 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 451
- 2024Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Modelscited by 301
- 2022Lost at C: A User Study on the Security Implications of Large Language Model Code Assistantscited by 87
- 2023A Survey on Large Language Models for Software Engineeringcited by 58
- 2025How Do AI Agents Do Human Work? Comparing AI and Human Workflows Across Diverse Occupationscited by 17
- 2022Using Deep Learning to Generate Complete Log Statementscited by 16
- 2024Unveiling ChatGPT's Usage in Open Source Projects: A Mining-based Studycited by 16
- 2024Software Engineering and Foundation Models: Insights from Industry Blogs Using a Jury of Foundation Modelscited by 6
- 2025Comparing Human and LLM Generated Code: The Jury is Still Out!cited by 6
3 more
- 2023Studying Logging Practice in Machine Learning-based Applicationscited by 3
- 2025Learning from Mistakes: Understanding Ad-hoc Logs through Analyzing Accidental Commitscited by 1
- 2025Automated File-Level Logging Generation for Machine Learning Applications using LLMs: A Case Study using GPT-4o Minicited by 1
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Humans are Missing from AI Coding Agent Research
- 2026SWE-chat: Coding Agent Interactions From Real Users in the Wild
- 2026Code Change Characteristics and Description Alignment: A Comparative Study of Agentic versus Human Pull Requests
- 2026Investigating Autonomous Agent Contributions in the Wild: Activity Patterns and Code Change over Time
- 2026Humans Integrate, Agents Fix: How Agent-Authored Pull Requests Are Referenced in Practice
- 2025Code with Me or for Me? How Increasing AI Automation Transforms Developer Workflows
- 2026Security in the Age of AI Teammates: An Empirical Study of Agentic Pull Requests on GitHub
- 2026How Do AI Coding Agents Contribute to Software Development? an Empirical Study of Agentic Pull Requests