# PTM4Tag: Sharpening Tag Recommendation of Stack Overflow Posts with Pre-trained Models

arXiv: https://arxiv.org/abs/2203.10965 · HTML: https://mlsanity.com/paper/2203.10965/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Mar 2022.

## Built on by (most cited first, 16 of 16)

- [Large Language Models for Software Engineering: A Systematic Literature Review](https://mlsanity.com/paper/2308.10620/) (2023)
- [How Can Recommender Systems Benefit from Large Language Models: A Survey](https://mlsanity.com/paper/2306.05817/) (2023)
- [When Large Language Models Meet Personalization: Perspectives of Challenges and Opportunities](https://mlsanity.com/paper/2307.16376/) (2023)
- [ReLLa: Retrieval-enhanced Large Language Models for Lifelong Sequential Behavior Comprehension in Recommendation](https://mlsanity.com/paper/2308.11131/) (2023)
- [Stealthy Backdoor Attack for Code Models](https://mlsanity.com/paper/2301.02496/) (2023)
- [History, Development, and Principles of Large Language Models-An Introductory Survey](https://mlsanity.com/paper/2402.06853/) (2024)
- [Deep Learning Model Reuse in the HuggingFace Community: Challenges, Benefit and Trends](https://mlsanity.com/paper/2401.13177/) (2024)
- [Large Language Models Make Sample-Efficient Recommender Systems](https://mlsanity.com/paper/2406.02368/) (2024)
- [Diverse Title Generation for Stack Overflow Posts with Multiple Sampling Enhanced Transformer](https://mlsanity.com/paper/2208.11523/) (2022)
- [Studying and Recommending Information Highlighting in Stack Overflow Answers](https://mlsanity.com/paper/2401.01472/) (2024)
- [Characterizing and Classifying Developer Forum Posts with their Intentions](https://mlsanity.com/paper/2312.14279/) (2023)
- [On Unified Prompt Tuning for Request Quality Assurance in Public Code Review](https://mlsanity.com/paper/2404.07942/) (2024)
- [StackRAG Agent: Improving Developer Answers with Retrieval-Augmented Generation](https://mlsanity.com/paper/2406.13840/) (2024)
- [Skill over Scale: The Case for Medium, Domain-Specific Models for SE](https://mlsanity.com/paper/2306.03268/) (2023)
- [Knowledge-Guided Prompt Learning for Request Quality Assurance in Public Code Review](https://mlsanity.com/paper/2410.21673/) (2024)
- [ArchISMiner: A Framework for Automatic Mining of Architectural Issue-Solution Pairs from Online Developer Communities](https://mlsanity.com/paper/2510.21966/) (2025)

## Abstract (arXiv)

Stack Overflow is often viewed as the most influential Software Question Answer (SQA) website with millions of programming-related questions and answers. Tags play a critical role in efficiently structuring the contents in Stack Overflow and are vital to support a range of site operations, e.g., querying relevant contents. Poorly selected tags often introduce extra noise and redundancy, which leads to tag synonym and tag explosion problems. Thus, an automated tag recommendation technique that can accurately recommend high-quality tags is desired to alleviate the problems mentioned above. Inspired by the recent success of pre-trained language models (PTMs) in natural language processing (NLP), we present PTM4Tag, a tag recommendation framework for Stack Overflow posts that utilize PTMs with a triplet architecture, which models the components of a post, i.e., Title, Description, and Code with independent language models. To the best of our knowledge, this is the first work that leverages PTMs in the tag recommendation task of SQA sites. We comparatively evaluate the performance of PTM4Tag based on five popular pre-trained models: BERT, RoBERTa, ALBERT, CodeBERT, and BERTOverflow. Our results show that leveraging the software engineering (SE) domain-specific PTM CodeBERT in PTM4Tag achieves the best performance among the five considered PTMs and outperforms the state-of-the-art deep learning (Convolutional Neural Network-based) approach by a large margin in terms of average Precision@k, Recall@k, and F1-score@k. We conduct an ablation study to quantify the contribution of a post's constituent components (Title, Description, and Code Snippets) to the performance of PTM4Tag. Our results show that Title is the most important in predicting the most relevant tags, and utilizing all the components achieves the best performance.