# Predicting the First Response Latency of Maintainers and Contributors in Pull Requests

arXiv: https://arxiv.org/abs/2311.07786 · HTML: https://mlsanity.com/paper/2311.07786/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 84 references that are in the map.
- First posted on arXiv 13 Nov 2023.

## Builds on (most cited first, 22 of 22)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously](https://mlsanity.com/paper/1801.01489/) (2018)
- [The Impact of Automated Parameter Optimization on Defect Prediction Models](https://mlsanity.com/paper/1801.10270/) (2018)
- [What's in a GitHub Star? Understanding Repository Starring Practices in a Social Coding Platform](https://mlsanity.com/paper/1811.07643/) (2018)
- [The impact of feature importance methods on the interpretation of defect classifiers](https://mlsanity.com/paper/2202.02389/) (2022)
- [A ground-truth dataset and classification model for detecting bots in GitHub issue and PR comments](https://mlsanity.com/paper/2010.03303/) (2020)
- [Does Code Quality Affect Pull Request Acceptance? An empirical study](https://mlsanity.com/paper/1908.09321/) (2019)
- [Quality Gatekeepers: Investigating the Effects ofCode Review Bots on Pull Request Activities](https://mlsanity.com/paper/2103.13547/) (2021)
- [Don't Disturb Me: Challenges of Interacting with SoftwareBots on Open Source Software Projects](https://mlsanity.com/paper/2103.13950/) (2021)
- [A Systematic Literature Review and Taxonomy of Modern Code Review](https://mlsanity.com/paper/2103.08777/) (2021)
- [Nudge: Accelerating Overdue Pull Requests Towards Completion](https://mlsanity.com/paper/2011.12468/) (2020)
- [Pull Request Latency Explained: An Empirical Overview](https://mlsanity.com/paper/2108.09946/) (2021)
- [On Wasted Contributions: Understanding the Dynamics of Contributor-Abandoned Pull Requests](https://mlsanity.com/paper/2110.15447/) (2021)
- [Predictive Models in Software Engineering: Challenges and Opportunities](https://mlsanity.com/paper/2008.03656/) (2020)
- [Bots Don't Mind Waiting, Do They? Comparing the Interaction With Automatically and Manually Created Pull Requests](https://mlsanity.com/paper/2103.03591/) (2021)
- [On the Relationship Between the Developer's Perceptible Race and Ethnicity and the Evaluation of Contributions in OSS](https://mlsanity.com/paper/2104.06143/) (2021)
- [Mining Code Review Data to Understand Waiting Times Between Acceptance and Merging: An Empirical Analysis](https://mlsanity.com/paper/2203.05048/) (2022)
- [How Successful Are Open Source Contributions From Countries with Different Levels of Human Development?](https://mlsanity.com/paper/2012.03716/) (2020)
- [Understanding the Time to First Response In GitHub Pull Requests](https://mlsanity.com/paper/2304.08426/) (2023)
- [Understanding the Helpfulness of Stale Bot for Pull-based Development: An Empirical Study of 20 Large Open-Source Projects](https://mlsanity.com/paper/2305.18150/) (2023)
- [Does the First Response Matter for Future Contributions? A Study of First Contributions](https://mlsanity.com/paper/2104.02933/) (2021)

## Built on by (most cited first, 3 of 3)

- [Automatic Detection of LLM-Generated Code: A Comparative Case Study of Contemporary Models Across Function and Class Granularities](https://mlsanity.com/paper/2409.01382/) (2024)
- [Synergizing LLMs and Knowledge Graphs: A Novel Approach to Software Repository-Related Question Answering](https://mlsanity.com/paper/2412.03815/) (2024)
- [An Approach for Auto Generation of Labeling Functions for Software Engineering Chatbots](https://mlsanity.com/paper/2410.07094/) (2024)

## Abstract (arXiv)

The success of a Pull Request (PR) depends on the responsiveness of the maintainers and the contributor during the review process. Being aware of the expected waiting times can lead to better interactions and managed expectations for both the maintainers and the contributor. In this paper, we propose a machine-learning approach to predict the first response latency of the maintainers following the submission of a PR, and the first response latency of the contributor after receiving the first response from the maintainers. We curate a dataset of 20 large and popular open-source projects on GitHub and extract 21 features to characterize projects, contributors, PRs, and review processes. Using these features, we then evaluate seven types of classifiers to identify the best-performing models. We also conduct permutation feature importance and SHAP analyses to understand the importance and the impact of different features on the predicted response latencies. We find that our CatBoost models are the most effective for predicting the first response latencies of both maintainers and contributors. We also observe that PRs submitted earlier in the week, containing an average number of commits, and with concise descriptions are more likely to receive faster first responses from the maintainers. Similarly, PRs with a lower first response latency from maintainers, that received the first response of maintainers earlier in the week, and containing an average number of commits tend to receive faster first responses from the contributors. Additionally, contributors with a higher acceptance rate and a history of timely responses in the project are likely to both obtain and provide faster first responses. Moreover, we show the effectiveness of our approach in a cross-project setting.