# Diverse Title Generation for Stack Overflow Posts with Multiple Sampling Enhanced Transformer

arXiv: https://arxiv.org/abs/2208.11523 · HTML: https://mlsanity.com/paper/2208.11523/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 84 references that are in the map.
- First posted on arXiv 24 Aug 2022.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation](https://mlsanity.com/paper/2109.00859/) (2021)
- [Measuring Coding Challenge Competence With APPS](https://mlsanity.com/paper/2105.09938/) (2021)
- [CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation](https://mlsanity.com/paper/2102.04664/) (2021)
- [InCoder: A Generative Model for Code Infilling and Synthesis](https://mlsanity.com/paper/2204.05999/) (2022)
- [Unified Pre-training for Program Understanding and Generation](https://mlsanity.com/paper/2103.06333/) (2021)
- [A Systematic Evaluation of Large Language Models of Code](https://mlsanity.com/paper/2202.13169/) (2022)
- [UniXcoder: Unified Cross-Modal Pre-training for Code Representation](https://mlsanity.com/paper/2203.03850/) (2022)
- [A Transformer-based Approach for Source Code Summarization](https://mlsanity.com/paper/2005.00653/) (2020)
- [Natural Language to Code Translation with Execution](https://mlsanity.com/paper/2204.11454/) (2022)
- [Sharp Nearby, Fuzzy Far Away: How Neural Language Models Use Context](https://mlsanity.com/paper/1805.04623/) (2018)
- [Modeling Coverage for Neural Machine Translation](https://mlsanity.com/paper/1601.04811/) (2016)
- [Applying CodeBERT for Automated Program Repair of Java Simple Bugs](https://mlsanity.com/paper/2103.11626/) (2021)
- [Fault-Aware Neural Code Rankers](https://mlsanity.com/paper/2206.03865/) (2022)
- [CAST: Enhancing Code Summarization with Hierarchical Splitting and Reconstruction of Abstract Syntax Trees](https://mlsanity.com/paper/2108.12987/) (2021)
- [A Multi-Modal Transformer-based Code Summarization Approach for Smart Contracts](https://mlsanity.com/paper/2103.07164/) (2021)
- [PTM4Tag: Sharpening Tag Recommendation of Stack Overflow Posts with Pre-trained Models](https://mlsanity.com/paper/2203.10965/) (2022)
- [AST-Transformer: Encoding Abstract Syntax Trees Efficiently for Code Summarization](https://mlsanity.com/paper/2112.01184/) (2021)
- [Generating Question Titles for Stack Overflow from Mined Code Snippets](https://mlsanity.com/paper/2005.10157/) (2020)
- [SOTorrent: Reconstructing and Analyzing the Evolution of Stack Overflow Posts](https://mlsanity.com/paper/1803.07311/) (2018)

## Built on by (most cited first, 2 of 2)

- [From Text Segmentation to Smart Chaptering: A Novel Benchmark for Structuring Video Transcriptions](https://mlsanity.com/paper/2402.17633/) (2024)
- [How to Generate Popular Post Headlines on Social Media?](https://mlsanity.com/paper/2309.09949/) (2023)

## Abstract (arXiv)

Stack Overflow is one of the most popular programming communities where developers can seek help for their encountered problems. Nevertheless, if inexperienced developers fail to describe their problems clearly, it is hard for them to attract sufficient attention and get the anticipated answers. We propose M3NSCT5, a novel approach to automatically generate multiple post titles from the given code snippets. Developers may use the generated titles to find closely related posts and complete their problem descriptions. M3NSCT5 employs the CodeT5 backbone, which is a pre-trained Transformer model having an excellent language understanding and generation ability. To alleviate the ambiguity issue that the same code snippets could be aligned with different titles under varying contexts, we propose the maximal marginal multiple nucleus sampling strategy to generate multiple high-quality and diverse title candidates at a time for the developers to choose from. We build a large-scale dataset with 890,000 question posts covering eight programming languages to validate the effectiveness of M3NSCT5. The automatic evaluation results on the BLEU and ROUGE metrics demonstrate the superiority of M3NSCT5 over six state-of-the-art baseline models. Moreover, a human evaluation with trustworthy results also demonstrates the great potential of our approach for real-world application.