Diverse Title Generation for Stack Overflow Posts with Multiple Sampling Enhanced Transformer
Not on the map. Why?
Abstract (arXiv)
Stack Overflow is one of the most popular programming communities where developers can seek help for their encountered problems. Nevertheless, if inexperienced developers fail to describe their problems clearly, it is hard for them to attract sufficient attention and get the anticipated answers. We propose M3NSCT5, a novel approach to automatically generate multiple post titles from the given code snippets. Developers may use the generated titles to find closely related posts and complete their problem descriptions. M3NSCT5 employs the CodeT5 backbone, which is a pre-trained Transformer model having an excellent language understanding and generation ability. To alleviate the ambiguity issue that the same code snippets could be aligned with different titles under varying contexts, we propose the maximal marginal multiple nucleus sampling strategy to generate multiple high-quality and diverse title candidates at a time for the developers to choose from. We build a large-scale dataset with 890,000 question posts covering eight programming languages to validate the effectiveness of M3NSCT5. The automatic evaluation results on the BLEU and ROUGE metrics demonstrate the superiority of M3NSCT5 over six state-of-the-art baseline models. Moreover, a human evaluation with trustworthy results also demonstrates the great potential of our approach for real-world application.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 34 of its 84 references that are in the map. First posted on arXiv 24 Aug 2022.
Builds on
34 of 84 references.
References · 34
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
- 2019The Curious Case of Neural Text Degenerationcited by 2,916
- 2022Competition-Level Code Generation with AlphaCodecited by 1,462
- 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,310
- 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,292
- 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 906
22 more
- 2021Measuring Coding Challenge Competence With APPScited by 902
- 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 744
- 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 498
- 2021Unified Pre-training for Program Understanding and Generationcited by 366
- 2022A Systematic Evaluation of Large Language Models of Codecited by 363
- 2022UniXcoder: Unified Cross-Modal Pre-training for Code Representationcited by 361
- 2020A Transformer-based Approach for Source Code Summarizationcited by 133
- 2022Natural Language to Code Translation with Executioncited by 124
- 2018Sharp Nearby, Fuzzy Far Away: How Neural Language Models Use Contextcited by 113
- 2016Modeling Coverage for Neural Machine Translationcited by 110
- 2021Applying CodeBERT for Automated Program Repair of Java Simple Bugscited by 44
- 2022Fault-Aware Neural Code Rankerscited by 33
- 2021CAST: Enhancing Code Summarization with Hierarchical Splitting and Reconstruction of Abstract Syntax Treescited by 32
- 2021A Multi-Modal Transformer-based Code Summarization Approach for Smart Contractscited by 16
- 2022PTM4Tag: Sharpening Tag Recommendation of Stack Overflow Posts with Pre-trained Modelscited by 16
- 2021AST-Transformer: Encoding Abstract Syntax Trees Efficiently for Code Summarizationcited by 11
- 2020Generating Question Titles for Stack Overflow from Mined Code Snippetscited by 10
- 2018SOTorrent: Reconstructing and Analyzing the Evolution of Stack Overflow Postscited by 9
- 2021Improving Stack Overflow question title generation with copying enhanced CodeBERT model and bi-modal informationcited by 5
- 2019Essential Sentences for Navigating Stack Overflow Answerscited by 4
- 2022SOTitle: A Transformer-based Post Title Generation Approach for Stack Overflowcited by 3
- 2020Characterizing Research Leadership on Geographical Weighted Collaboration Networkcited by 2
Built on by
Citing papers per year: 2023: 1; 2024: 1.
2 papers.