# Convex Aggregation for Opinion Summarization

arXiv: https://arxiv.org/abs/2104.01371 · HTML: https://mlsanity.com/paper/2104.01371/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 54 references that are in the map.
- First posted on arXiv 3 Apr 2021.

## Builds on (most cited first, 30 of 35)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [On the Convergence of Adam and Beyond](https://mlsanity.com/paper/1904.09237/) (2019)
- [Improving Variational Inference with Inverse Autoregressive Flow](https://mlsanity.com/paper/1606.04934/) (2016)
- [Generating Sentences from a Continuous Space](https://mlsanity.com/paper/1511.06349/) (2015)
- [Self-critical Sequence Training for Image Captioning](https://mlsanity.com/paper/1612.00563/) (2016)
- [Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering](https://mlsanity.com/paper/1602.01585/) (2016)
- [A Neural Attention Model for Abstractive Sentence Summarization](https://mlsanity.com/paper/1509.00685/) (2015)
- [A Deep Reinforced Model for Abstractive Summarization](https://mlsanity.com/paper/1705.04304/) (2017)
- [Using the Output Embedding to Improve Language Models](https://mlsanity.com/paper/1608.05859/) (2016)
- [Toward Controlled Generation of Text](https://mlsanity.com/paper/1703.00955/) (2017)
- [Cyclical Annealing Schedule: A Simple Approach to Mitigating KL Vanishing](https://mlsanity.com/paper/1903.10145/) (2019)
- [Show Your Work: Improved Reporting of Experimental Results](https://mlsanity.com/paper/1909.03004/) (2019)
- [Hierarchical Transformers for Multi-Document Summarization](https://mlsanity.com/paper/1905.13164/) (2019)
- [Disentangled Representation Learning for Non-Parallel Text Style Transfer](https://mlsanity.com/paper/1808.04339/) (2018)
- [Neural Summarization by Extracting Sentences and Words](https://mlsanity.com/paper/1603.07252/) (2016)
- [Optimus: Organizing Sentences via Pre-trained Modeling of a Latent Space](https://mlsanity.com/paper/2004.04092/) (2020)
- [Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling](https://mlsanity.com/paper/1611.01462/) (2016)
- [Extractive Opinion Summarization in Quantized Transformer Spaces](https://mlsanity.com/paper/2012.04443/) (2020)
- [Summarizing Opinions: Aspect Extraction Meets Sentiment Prediction and They Are Both Weakly Supervised](https://mlsanity.com/paper/1808.08858/) (2018)
- [Unsupervised Opinion Summarization as Copycat-Review Generation](https://mlsanity.com/paper/1911.02247/) (2019)
- [Generating Sentences from Disentangled Syntactic and Semantic Spaces](https://mlsanity.com/paper/1907.05789/) (2019)
- [What Kind of Language Is Hard to Language-Model?](https://mlsanity.com/paper/1906.04726/) (2019)
- [Unsupervised Opinion Summarization with Noising and Denoising](https://mlsanity.com/paper/2004.10150/) (2020)

## Built on by (most cited first, 21 of 21)

- [Comparative Opinion Summarization via Collaborative Decoding](https://mlsanity.com/paper/2110.07520/) (2021)
- [Unsupervised Extractive Opinion Summarization Using Sparse Coding](https://mlsanity.com/paper/2203.07921/) (2022)
- [Factual and Informative Review Generation for Explainable Recommendation](https://mlsanity.com/paper/2209.12613/) (2022)
- [Scientific Opinion Summarization: Paper Meta-review Generation Dataset, Methods, and Evaluation](https://mlsanity.com/paper/2305.14647/) (2023)
- [Beyond Opinion Mining: Summarizing Opinions of Customer Reviews](https://mlsanity.com/paper/2206.01543/) (2022)
- [Factorizing Content and Budget Decisions in Abstractive Summarization of Long Documents](https://mlsanity.com/paper/2205.12486/) (2022)
- [Hierarchical Indexing for Retrieval-Augmented Opinion Summarization](https://mlsanity.com/paper/2403.00435/) (2024)
- [Attributable and Scalable Opinion Summarization](https://mlsanity.com/paper/2305.11603/) (2023)
- [Unsupervised Opinion Summarization Using Approximate Geodesics](https://mlsanity.com/paper/2209.07496/) (2022)
- [Absformer: Transformer-based Model for Unsupervised Multi-Document Abstractive Summarization](https://mlsanity.com/paper/2306.04787/) (2023)
- [Rationale-based Opinion Summarization](https://mlsanity.com/paper/2404.00217/) (2024)
- [Unsupervised Opinion Summarisation in the Wasserstein Space](https://mlsanity.com/paper/2211.14923/) (2022)
- [Simple Yet Effective Synthetic Dataset Construction for Unsupervised Opinion Summarization](https://mlsanity.com/paper/2303.11660/) (2023)
- [Rehearse With User: Personalized Opinion Summarization via Role-Playing based on Large Language Models](https://mlsanity.com/paper/2503.00449/) (2025)
- [Automatically Evaluating Opinion Prevalence in Opinion Summarization](https://mlsanity.com/paper/2307.14305/) (2023)
- [TL;DR Progress: Multi-faceted Literature Exploration in Text Summarization](https://mlsanity.com/paper/2402.06913/) (2024)
- [Coverage-based Fairness in Multi-document Summarization](https://mlsanity.com/paper/2412.08795/) (2024)
- [Learning Cluster Patterns for Abstractive Summarization](https://mlsanity.com/paper/2202.10967/) (2022)
- [Summary Workbench: Unifying Application and Evaluation of Text Summarization Models](https://mlsanity.com/paper/2210.09587/) (2022)
- [Towards an automatic method for generating topical vocabulary test forms for specific reading passages](https://mlsanity.com/paper/2505.18762/) (2025)
- [Graph Laplacian Transformer with Progressive Sampling for Prostate Cancer Grading](https://mlsanity.com/paper/2512.10808/) (2025)

## Abstract (arXiv)

Recent advances in text autoencoders have significantly improved the quality of the latent space, which enables models to generate grammatical and consistent text from aggregated latent vectors. As a successful application of this property, unsupervised opinion summarization models generate a summary by decoding the aggregated latent vectors of inputs. More specifically, they perform the aggregation via simple average. However, little is known about how the vector aggregation step affects the generation quality. In this study, we revisit the commonly used simple average approach by examining the latent space and generated summaries. We found that text autoencoders tend to generate overly generic summaries from simply averaged latent vectors due to an unexpected L2-norm shrinkage in the aggregated latent vectors, which we refer to as summary vector degeneration. To overcome this issue, we develop a framework Coop, which searches input combinations for the latent vector aggregation using input-output word overlap. Experimental results show that Coop successfully alleviates the summary vector degeneration issue and establishes new state-of-the-art performance on two opinion summarization benchmarks. Code is available at urlhttps://github.com/megagonlabs/coop.