# BotHawk: An Approach for Bots Detection in Open Source Software Projects

arXiv: https://arxiv.org/abs/2307.13386 · HTML: https://mlsanity.com/paper/2307.13386/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 41 references that are in the map.
- First posted on arXiv 25 Jul 2023.

## Builds on (most cited first, 9 of 9)

- [Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions](https://mlsanity.com/paper/2108.09293/) (2021)
- [The spread of low-credibility content by social bots](https://mlsanity.com/paper/1707.07592/) (2017)
- [Simplistic Collection and Labeling Practices Limit the Utility of Benchmark Datasets for Twitter Bot Detection](https://mlsanity.com/paper/2301.07015/) (2023)
- [Detecting and Characterizing Bots that Commit Code](https://mlsanity.com/paper/2003.03172/) (2020)
- [A ground-truth dataset and classification model for detecting bots in GitHub issue and PR comments](https://mlsanity.com/paper/2010.03303/) (2020)
- [Measuring bot and human behavioral dynamics](https://mlsanity.com/paper/1802.04286/) (2018)
- [Patterns of Effort Contribution and Demand and User Classification based on Participation Patterns in NPM Ecosystem](https://mlsanity.com/paper/1907.06538/) (2019)
- [Identifying bot activity in GitHub pull request and issue comments](https://mlsanity.com/paper/2103.06042/) (2021)
- [Evolving Collaboration, Dependencies, and Use in the Rust Open Source Software Ecosystem](https://mlsanity.com/paper/2205.03597/) (2022)

## Built on by (most cited first, 1 of 1)

- [DevBots can co-design APIs](https://mlsanity.com/paper/2312.05733/) (2023)

## Abstract (arXiv)

Social coding platforms have revolutionized collaboration in software development, leading to using software bots for streamlining operations. However, The presence of open-source software (OSS) bots gives rise to problems including impersonation, spamming, bias, and security risks. Identifying bot accounts and behavior is a challenging task in the OSS project. This research aims to investigate bots' behavior in open-source software projects and identify bot accounts with maximum possible accuracy. Our team gathered a dataset of 19,779 accounts that meet standardized criteria to enable future research on bots in open-source projects. We follow a rigorous workflow to ensure that the data we collect is accurate, generalizable, scalable, and up-to-date. We've identified four types of bot accounts in open-source software projects by analyzing their behavior across 17 features in 5 dimensions. Our team created BotHawk, a highly effective model for detecting bots in open-source software projects. It outperforms other models, achieving an AUC of 0.947 and an F1-score of 0.89. BotHawk can detect a wider variety of bots, including CI/CD and scanning bots. Furthermore, we find that the number of followers, number of repositories, and tags contain the most relevant features to identify the account type.