Predicting Issue Types with seBERT
Not on the map. Why?
Abstract (arXiv)
Pre-trained transformer models are the current state-of-the-art for natural language models processing. seBERT is such a model, that was developed based on the BERT architecture, but trained from scratch with software engineering data. We fine-tuned this model for the NLBSE challenge for the task of issue type prediction. Our model dominates the baseline fastText for all three issue types in both recall and precisio} to achieve an overall F1-score of 85.7%, which is an increase of 4.1% over the baseline.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 6 of its 13 references that are in the map. First posted on arXiv 3 May 2022.
Builds on
6 of 13 references.
References · 6
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020Big Bird: Transformers for Longer Sequencescited by 1,684
- 2021On the validity of pre-trained transformers for natural language processing in the software engineering domaincited by 11
- 2021Predicting Issue Types on GitHubcited by 10
- 2020On the feasibility of automated prediction of bug and non-bug issuescited by 6
Built on by
Citing papers per year: 2022: 1; 2023: 0; 2024: 0; 2025: 1.
2 papers.