# Generation of Highlights from Research Papers Using Pointer-Generator Networks and SciBERT Embeddings

arXiv: https://arxiv.org/abs/2302.07729 · HTML: https://mlsanity.com/paper/2302.07729/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 66 references that are in the map.
- First posted on arXiv 14 Feb 2023.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond](https://mlsanity.com/paper/1602.06023/) (2016)
- [SummEval: Re-evaluating Summarization Evaluation](https://mlsanity.com/paper/2007.12626/) (2020)
- [Carbon Emissions and Large Neural Network Training](https://mlsanity.com/paper/2104.10350/) (2021)
- [ERNIE: Enhanced Language Representation with Informative Entities](https://mlsanity.com/paper/1905.07129/) (2019)
- [Model Evaluation, Model Selection, and Algorithm Selection in Machine Learning](https://mlsanity.com/paper/1811.12808/) (2018)
- [Construction of the Literature Graph in Semantic Scholar](https://mlsanity.com/paper/1805.02262/) (2018)
- [Modeling Coverage for Neural Machine Translation](https://mlsanity.com/paper/1601.04811/) (2016)
- [Growth rates of modern science: A latent piecewise growth curve approach to model publication numbers from established and new literature databases](https://mlsanity.com/paper/2012.07675/) (2020)
- [A Supervised Approach to Extractive Summarisation of Scientific Papers](https://mlsanity.com/paper/1706.03946/) (2017)
- [Abstractive Text Summarization based on Language Model Conditioning and Locality Modeling](https://mlsanity.com/paper/2003.13027/) (2020)
- [Data-driven Summarization of Scientific Articles](https://mlsanity.com/paper/1804.08875/) (2018)
- [Towards Abstraction from Extraction: Multiple Timescale Gated Recurrent Unit for Summarization](https://mlsanity.com/paper/1607.00718/) (2016)

## Built on by (most cited first, 12 of 12)

- [Hallucination Reduction in Long Input Text Summarization](https://mlsanity.com/paper/2309.16781/) (2023)
- [Can pre-trained language models generate titles for research papers?](https://mlsanity.com/paper/2409.14602/) (2024)
- [Evaluating LLMs and Pre-trained Models for Text Summarization Across Diverse Datasets](https://mlsanity.com/paper/2502.19339/) (2025)
- [Analysis of Multidomain Abstractive Summarization Using Salience Allocation](https://mlsanity.com/paper/2402.11955/) (2024)
- [How Green are Neural Language Models? Analyzing Energy Consumption in Text Summarization Fine-tuning](https://mlsanity.com/paper/2501.15398/) (2025)
- [Introducing Spotlight: A Novel Approach for Generating Captivating Key Information from Documents](https://mlsanity.com/paper/2509.10935/) (2025)
- [Comparative Analysis of Abstractive Summarization Models for Clinical Radiology Reports](https://mlsanity.com/paper/2506.16247/) (2025)
- [Enhancing Business Analytics through Hybrid Summarization of Financial Reports](https://mlsanity.com/paper/2601.09729/) (2025)
- [Overview of the SciHigh Track at FIRE 2025: Research Highlight Generation from Scientific Papers](https://mlsanity.com/paper/2601.11582/) (2026)
- [Enhancing Unsupervised Keyword Extraction in Academic Papers through Integrating Highlights with Abstract](https://mlsanity.com/paper/2604.19505/) (2026)
- [Automatic Generation of Titles for Research Papers Using Language Models](https://mlsanity.com/paper/2606.05085/) (2026)
- [Automatic Generation of Highlights for Academic Paper Via Prompt-based Learning](https://mlsanity.com/paper/2606.25253/) (2026)

## Abstract (arXiv)

Nowadays many research articles are prefaced with research highlights to summarize the main findings of the paper. Highlights not only help researchers precisely and quickly identify the contributions of a paper, they also enhance the discoverability of the article via search engines. We aim to automatically construct research highlights given certain segments of a research paper. We use a pointer-generator network with coverage mechanism and a contextual embedding layer at the input that encodes the input tokens into SciBERT embeddings. We test our model on a benchmark dataset, CSPubSum, and also present MixSub, a new multi-disciplinary corpus of papers for automatic research highlight generation. For both CSPubSum and MixSub, we have observed that the proposed model achieves the best performance compared to related variants and other models proposed in the literature. On the CSPubSum dataset, our model achieves the best performance when the input is only the abstract of a paper as opposed to other segments of the paper. It produces ROUGE-1, ROUGE-2 and ROUGE-L F1-scores of 38.26, 14.26 and 35.51, respectively, METEOR score of 32.62, and BERTScore F1 of 86.65 which outperform all other baselines. On the new MixSub dataset, where only the abstract is the input, our proposed model (when trained on the whole training corpus without distinguishing between the subject categories) achieves ROUGE-1, ROUGE-2 and ROUGE-L F1-scores of 31.78, 9.76 and 29.3, respectively, METEOR score of 24.00, and BERTScore F1 of 85.25.