# Security Bug Report Prediction Within and Across Projects: A Comparative Study of BERT and Random Forest

arXiv: https://arxiv.org/abs/2504.21037 · HTML: https://mlsanity.com/paper/2504.21037/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 26 references that are in the map.
- First posted on arXiv 28 Apr 2025.

## Builds on (most cited first, 8 of 8)

- [Unifying Large Language Models and Knowledge Graphs: A Roadmap](https://mlsanity.com/paper/2306.08302/) (2023)
- [Learning to Reduce False Positives in Analytic Bug Detectors](https://mlsanity.com/paper/2203.09907/) (2022)
- [Benchmarking Prompt Engineering Techniques for Secure Code Generation with GPT Models](https://mlsanity.com/paper/2502.06039/) (2025)
- [Security Smells in Android](https://mlsanity.com/paper/2006.01181/) (2020)
- [How Do Developers Deal with Security Issue Reports on GitHub?](https://mlsanity.com/paper/2112.10359/) (2021)
- [Better Security Bug Report Classification via Hyperparameter Optimization](https://mlsanity.com/paper/1905.06872/) (2019)
- [Security Risks of Porting C Programs to WebAssembly](https://mlsanity.com/paper/2112.11745/) (2021)
- [Insecure by Design in the Backbone of Critical Infrastructure](https://mlsanity.com/paper/2303.12340/) (2023)

## Built on by (most cited first, 2 of 2)

- [Evaluating Large Language Models for Security Bug Report Prediction](https://mlsanity.com/paper/2601.22921/) (2026)
- [From Data Leak to Secret Misses: The Impact of Data Leakage on Secret Detection Models](https://mlsanity.com/paper/2601.22946/) (2026)

## Abstract (arXiv)

Early detection of security bug reports (SBRs) is crucial for preventing vulnerabilities and ensuring system reliability. While machine learning models have been developed for SBR prediction, their predictive performance still has room for improvement. In this study, we conduct a comprehensive comparison between BERT and Random Forest (RF), a competitive baseline for predicting SBRs. The results show that RF outperforms BERT with a 34% higher average G-measure for within-project predictions. Adding only SBRs from various projects improves both models' average performance. However, including both security and nonsecurity bug reports significantly reduces RF's average performance to 46%, while boosts BERT to its best average performance of 66%, surpassing RF. In cross-project SBR prediction, BERT achieves a remarkable 62% G-measure, which is substantially higher than RF.