# Unsupervised Abstractive Opinion Summarization by Generating Sentences with Tree-Structured Topic Guidance

arXiv: https://arxiv.org/abs/2106.08007 · HTML: https://mlsanity.com/paper/2106.08007/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 57 references that are in the map.
- First posted on arXiv 15 Jun 2021.

## Builds on (most cited first, 30 of 33)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluation](https://mlsanity.com/paper/1708.00055/) (2017)
- [On Faithfulness and Factuality in Abstractive Summarization](https://mlsanity.com/paper/2005.00661/) (2020)
- [Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond](https://mlsanity.com/paper/1602.06023/) (2016)
- [Image-based Recommendations on Styles and Substitutes](https://mlsanity.com/paper/1506.04757/) (2015)
- [Generating Sentences from a Continuous Space](https://mlsanity.com/paper/1511.06349/) (2015)
- [Text Summarization with Pretrained Encoders](https://mlsanity.com/paper/1908.08345/) (2019)
- [Texygen: A Benchmarking Platform for Text Generation Models](https://mlsanity.com/paper/1802.01886/) (2018)
- [Multi-News: a Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Model](https://mlsanity.com/paper/1906.01749/) (2019)
- [Toward Controlled Generation of Text](https://mlsanity.com/paper/1703.00955/) (2017)
- [Hierarchical Transformers for Multi-Document Summarization](https://mlsanity.com/paper/1905.13164/) (2019)
- [Word Representations via Gaussian Embedding](https://mlsanity.com/paper/1412.6623/) (2014)
- [Improved Variational Autoencoders for Text Modeling using Dilated Convolutions](https://mlsanity.com/paper/1702.08139/) (2017)
- [Discovering Discrete Latent Topics with Neural Variational Inference](https://mlsanity.com/paper/1706.00359/) (2017)
- [Extractive Opinion Summarization in Quantized Transformer Spaces](https://mlsanity.com/paper/2012.04443/) (2020)
- [Summarizing Opinions: Aspect Extraction Meets Sentiment Prediction and They Are Both Weakly Supervised](https://mlsanity.com/paper/1808.08858/) (2018)
- [Unsupervised Opinion Summarization as Copycat-Review Generation](https://mlsanity.com/paper/1911.02247/) (2019)
- [Topic-Guided Variational Autoencoders for Text Generation](https://mlsanity.com/paper/1903.07137/) (2019)
- [Unsupervised Opinion Summarization with Noising and Denoising](https://mlsanity.com/paper/2004.10150/) (2020)
- [SEQ^3: Differentiable Sequence-to-Sequence-to-Sequence Autoencoder for Unsupervised Abstractive Sentence Compression](https://mlsanity.com/paper/1904.03651/) (2019)
- [Capturing Reliable Fine-Grained Sentiment Associations by Crowdsourcing and Best-Worst Scaling](https://mlsanity.com/paper/1712.01741/) (2017)

## Built on by (most cited first, 14 of 14)

- [Prompted Opinion Summarization with GPT-3.5](https://mlsanity.com/paper/2211.15914/) (2022)
- [Comparative Opinion Summarization via Collaborative Decoding](https://mlsanity.com/paper/2110.07520/) (2021)
- [Efficient Few-Shot Fine-Tuning for Opinion Summarization](https://mlsanity.com/paper/2205.02170/) (2022)
- [Scientific Opinion Summarization: Paper Meta-review Generation Dataset, Methods, and Evaluation](https://mlsanity.com/paper/2305.14647/) (2023)
- [OpinSummEval: Revisiting Automated Evaluation for Opinion Summarization](https://mlsanity.com/paper/2310.18122/) (2023)
- [Large-Scale and Multi-Perspective Opinion Summarization with Diverse Review Subsets](https://mlsanity.com/paper/2310.13340/) (2023)
- [Beyond Opinion Mining: Summarizing Opinions of Customer Reviews](https://mlsanity.com/paper/2206.01543/) (2022)
- [Template-based Abstractive Microblog Opinion Summarisation](https://mlsanity.com/paper/2208.04083/) (2022)
- [Incremental Extractive Opinion Summarization Using Cover Trees](https://mlsanity.com/paper/2401.08047/) (2024)
- [OpineSum: Entailment-based self-training for abstractive opinion summarization](https://mlsanity.com/paper/2212.10791/) (2022)
- [Absformer: Transformer-based Model for Unsupervised Multi-Document Abstractive Summarization](https://mlsanity.com/paper/2306.04787/) (2023)
- [Rationale-based Opinion Summarization](https://mlsanity.com/paper/2404.00217/) (2024)
- [Simple Yet Effective Synthetic Dataset Construction for Unsupervised Opinion Summarization](https://mlsanity.com/paper/2303.11660/) (2023)
- [Automatically Evaluating Opinion Prevalence in Opinion Summarization](https://mlsanity.com/paper/2307.14305/) (2023)

## Abstract (arXiv)

This paper presents a novel unsupervised abstractive summarization method for opinionated texts. While the basic variational autoencoder-based models assume a unimodal Gaussian prior for the latent code of sentences, we alternate it with a recursive Gaussian mixture, where each mixture component corresponds to the latent code of a topic sentence and is mixed by a tree-structured topic distribution. By decoding each Gaussian component, we generate sentences with tree-structured topic guidance, where the root sentence conveys generic content, and the leaf sentences describe specific topics. Experimental results demonstrate that the generated topic sentences are appropriate as a summary of opinionated texts, which are more informative and cover more input contents than those generated by the recent unsupervised summarization model (Bravzinskas et al., 2020). Furthermore, we demonstrate that the variance of latent Gaussians represents the granularity of sentences, analogous to Gaussian word embedding (Vilnis and McCallum, 2015).