# Neural Abstractive Unsupervised Summarization of Online News Discussions

arXiv: https://arxiv.org/abs/2106.03953 · HTML: https://mlsanity.com/paper/2106.03953/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 24 references that are in the map.
- First posted on arXiv 7 Jun 2021.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Text Summarization with Pretrained Encoders](https://mlsanity.com/paper/1908.08345/) (2019)
- [Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation](https://mlsanity.com/paper/1611.04558/) (2016)
- [OpenNMT: Open-source Toolkit for Neural Machine Translation](https://mlsanity.com/paper/1709.03815/) (2017)
- [Neural Text Summarization: A Critical Evaluation](https://mlsanity.com/paper/1908.08960/) (2019)
- [Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting](https://mlsanity.com/paper/1805.11080/) (2018)
- [Abstractive Summarization of Reddit Posts with Multi-level Memory Networks](https://mlsanity.com/paper/1811.00783/) (2018)
- [On Extractive and Abstractive Neural Document Summarization with Transformer Language Models](https://mlsanity.com/paper/1909.03186/) (2019)
- [Neural Abstractive Text Summarization with Sequence-to-Sequence Models](https://mlsanity.com/paper/1812.02303/) (2018)
- [A Reinforced Topic-Aware Convolutional Sequence-to-Sequence Model for Abstractive Text Summarization](https://mlsanity.com/paper/1805.03616/) (2018)
- [Selective Encoding for Abstractive Sentence Summarization](https://mlsanity.com/paper/1704.07073/) (2017)
- [Global Encoding for Abstractive Summarization](https://mlsanity.com/paper/1805.03989/) (2018)
- [Abstractive Text Summarization by Incorporating Reader Comments](https://mlsanity.com/paper/1812.05407/) (2018)
- [Iterative Document Representation Learning Towards Summarization with Polishing](https://mlsanity.com/paper/1809.10324/) (2018)
- [Sequential Copying Networks](https://mlsanity.com/paper/1807.02301/) (2018)

## Built on by (most cited first, 1 of 1)

- [Absformer: Transformer-based Model for Unsupervised Multi-Document Abstractive Summarization](https://mlsanity.com/paper/2306.04787/) (2023)

## Abstract (arXiv)

Summarization has usually relied on gold standard summaries to train extractive or abstractive models. Social media brings a hurdle to summarization techniques since it requires addressing a multi-document multi-author approach. We address this challenging task by introducing a novel method that generates abstractive summaries of online news discussions. Our method extends a BERT-based architecture, including an attention encoding that fed comments' likes during the training stage. To train our model, we define a task which consists of reconstructing high impact comments based on popularity (likes). Accordingly, our model learns to summarize online discussions based on their most relevant comments. Our novel approach provides a summary that represents the most relevant aspects of a news item that users comment on, incorporating the social context as a source of information to summarize texts in online social networks. Our model is evaluated using ROUGE scores between the generated summary and each comment on the thread. Our model, including the social attention encoding, significantly outperforms both extractive and abstractive summarization methods based on such evaluation.