# Automatic Pull Request Description Generation Using LLMs: A T5 Model Approach

arXiv: https://arxiv.org/abs/2408.00921 · HTML: https://mlsanity.com/paper/2408.00921/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 21 references that are in the map.
- First posted on arXiv 1 Aug 2024.

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [A Deep Reinforced Model for Abstractive Summarization](https://mlsanity.com/paper/1705.04304/) (2017)
- [Long Short-Term Memory-Networks for Machine Reading](https://mlsanity.com/paper/1601.06733/) (2016)
- [Automatically Generating Commit Messages from Diffs using Neural Machine Translation](https://mlsanity.com/paper/1708.09492/) (2017)
- [CommitBERT: Commit Message Generation Using Pre-Trained Programming Language Model](https://mlsanity.com/paper/2105.14242/) (2021)
- [Automatic Generation of Pull Request Descriptions](https://mlsanity.com/paper/1909.06987/) (2019)
- [An Insight into the Pull Requests of GitHub](https://mlsanity.com/paper/1807.01853/) (2018)

## Built on by (most cited first, 2 of 2)

- [Code Change Characteristics and Description Alignment: A Comparative Study of Agentic versus Human Pull Requests](https://mlsanity.com/paper/2601.17627/) (2026)
- [Rethinking Code Review in the Age of AI: A Vision for Agentic Code Review](https://mlsanity.com/paper/2605.17548/) (2026)

## Abstract (arXiv)

Developers create pull request (PR) descriptions to provide an overview of their changes and explain the motivations behind them. These descriptions help reviewers and fellow developers quickly understand the updates. Despite their importance, some developers omit these descriptions. To tackle this problem, we propose an automated method for generating PR descriptions based on commit messages and source code comments. This method frames the task as a text summarization problem, for which we utilized the T5 text-to-text transfer model. We fine-tuned a pre-trained T5 model using a dataset containing 33,466 PRs. The model's effectiveness was assessed using ROUGE metrics, which are recognized for their strong alignment with human evaluations. Our findings reveal that the T5 model significantly outperforms LexRank, which served as our baseline for comparison.