# Detecting and Characterizing Bots that Commit Code

arXiv: https://arxiv.org/abs/2003.03172 · HTML: https://mlsanity.com/paper/2003.03172/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Mar 2020.

## Built on by (most cited first, 10 of 10)

- [From Commit Message Generation to History-Aware Commit Message Completion](https://mlsanity.com/paper/2308.07655/) (2023)
- [Motivations, Challenges, Best Practices, and Benefits for Bots and Conversational Agents in Software Engineering: A Multivocal Literature Review](https://mlsanity.com/paper/2409.11864/) (2024)
- [AgentPack: A Dataset of Code Changes, Co-Authored by Agents and Humans](https://mlsanity.com/paper/2509.21891/) (2025)
- [BotHawk: An Approach for Bots Detection in Open Source Software Projects](https://mlsanity.com/paper/2307.13386/) (2023)
- [The Sensitivity of Word Embeddings-based Author Detection Models to Semantic-preserving Adversarial Perturbations](https://mlsanity.com/paper/2102.11917/) (2021)
- [DevBots can co-design APIs](https://mlsanity.com/paper/2312.05733/) (2023)
- [Beyond Code Contributions: How Network Position, Temporal Bursts, and Code Review Activities Shape Contributor Influence in Large-Scale Open Source Ecosystems](https://mlsanity.com/paper/2602.06426/) (2026)
- [Before the Pull Request: Mining Multi-Agent Coordination](https://mlsanity.com/paper/2606.19616/) (2026)
- [Detecting AI Coding Agents in Open Source: A Validated Multi-Method Census of 180 Million Repositories](https://mlsanity.com/paper/2606.24429/) (2026)
- [CRJudgeBench: Can AI Detect Plausible but Invalid Code Reviews?](https://mlsanity.com/paper/2609.37216/) (2026)

## Abstract (arXiv)

Background: Some developer activity traditionally performed manually, such as making code commits, opening, managing, or closing issues is increasingly subject to automation in many OSS projects. Specifically, such activity is often performed by tools that react to events or run at specific times. We refer to such automation tools as bots and, in many software mining scenarios related to developer productivity or code quality it is desirable to identify bots in order to separate their actions from actions of individuals. Aim: Find an automated way of identifying bots and code committed by these bots, and to characterize the types of bots based on their activity patterns. Method and Result: We propose BIMAN, a systematic approach to detect bots using author names, commit messages, files modified by the commit, and projects associated with the ommits. For our test data, the value for AUC-ROC was 0.9. We also characterized these bots based on the time patterns of their code commits and the types of files modified, and found that they primarily work with documentation files and web pages, and these files are most prevalent in HTML and JavaScript ecosystems. We have compiled a shareable dataset containing detailed information about 461 bots we found (all of whom have more than 1000 commits) and 13,762,430 commits they created.