# Mixture Content Selection for Diverse Sequence Generation

arXiv: https://arxiv.org/abs/1909.01953 · HTML: https://mlsanity.com/paper/1909.01953/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Sep 2019.

## Built on by (most cited first, 27 of 27)

- [DOC2PPT: Automatic Presentation Slides Generation from Scientific Documents](https://mlsanity.com/paper/2101.11796/) (2021)
- [Text Generation by Learning from Demonstrations](https://mlsanity.com/paper/2009.07839/) (2020)
- [Mixed SIGNals: Sign Language Production via a Mixture of Motion Primitives](https://mlsanity.com/paper/2107.11317/) (2021)
- [A Survey of Multi-task Learning in Natural Language Processing: Regarding Task Relatedness and Training Methods](https://mlsanity.com/paper/2204.03508/) (2022)
- [Controllable Open-ended Question Generation with A New Question Type Ontology](https://mlsanity.com/paper/2107.00152/) (2021)
- [Universal Simultaneous Machine Translation with Mixture-of-Experts Wait-k Policy](https://mlsanity.com/paper/2109.05238/) (2021)
- [Focus Attention: Promoting Faithfulness and Diversity in Summarization](https://mlsanity.com/paper/2105.11921/) (2021)
- [A Well-Composed Text is Half Done! Composition Sampling for Diverse Conditional Generation](https://mlsanity.com/paper/2203.15108/) (2022)
- [Distractor Generation in Multiple-Choice Tasks: A Survey of Methods, Datasets, and Evaluation](https://mlsanity.com/paper/2402.01512/) (2024)
- [Diversifying Content Generation for Commonsense Reasoning with Mixture of Knowledge Graph Experts](https://mlsanity.com/paper/2203.07285/) (2022)
- [Diverse Text Generation via Variational Encoder-Decoder Models with Gaussian Process Priors](https://mlsanity.com/paper/2204.01227/) (2022)
- [Few-Shot Anaphora Resolution in Scientific Protocols via Mixtures of In-Context Experts](https://mlsanity.com/paper/2210.03690/) (2022)
- [Diversify Question Generation with Retrieval-Augmented Style Transfer](https://mlsanity.com/paper/2310.14503/) (2023)
- [TegTok: Augmenting Text Generation via Task-specific and Open-world Knowledge](https://mlsanity.com/paper/2203.08517/) (2022)
- [Simple or Complex? Complexity-Controllable Question Generation with Soft Templates and Deep Mixture of Experts Model](https://mlsanity.com/paper/2110.06560/) (2021)
- ["What makes a question inquisitive?" A Study on Type-Controlled Inquisitive Question Generation](https://mlsanity.com/paper/2205.08056/) (2022)
- [Sentence-Permuted Paragraph Generation](https://mlsanity.com/paper/2104.07228/) (2021)
- [Collaboration and Controversy Among Experts: Rumor Early Detection by Tuning a Comment Generator](https://mlsanity.com/paper/2504.04076/) (2025)
- [Reference-based Metrics Disprove Themselves in Question Generation](https://mlsanity.com/paper/2403.12242/) (2024)
- [Towards More Robust Natural Language Understanding](https://mlsanity.com/paper/2112.02992/) (2021)
- [KCS: Diversify Multi-hop Question Generation with Knowledge Composition Sampling](https://mlsanity.com/paper/2508.20567/) (2025)
- [CUSTOM: Aspect-Oriented Product Summarization for E-Commerce](https://mlsanity.com/paper/2108.08010/) (2021)
- [Learning Mixture Structure on Multi-Source Time Series for Probabilistic Forecasting](https://mlsanity.com/paper/2302.11078/) (2023)
- [RepCali: High Efficient Fine-tuning Via Representation Calibration in Latent Space for Pre-trained Language Models](https://mlsanity.com/paper/2505.08463/) (2025)
- [From Answers to Questions: EQGBench for Evaluating LLMs' Educational Question Generation](https://mlsanity.com/paper/2508.10005/) (2025)
- [Exploring Answer Information Methods for Question Generation with Transformers](https://mlsanity.com/paper/2312.03483/) (2023)
- [JaccDiv: A Metric and Benchmark for Quantifying Diversity of Generated Marketing Text in the Music Industry](https://mlsanity.com/paper/2504.20849/) (2025)

## Abstract (arXiv)

Generating diverse sequences is important in many NLP applications such as question generation or summarization that exhibit semantically one-to-many relationships between source and the target sequences. We present a method to explicitly separate diversification from generation using a general plug-and-play module (called SELECTOR) that wraps around and guides an existing encoder-decoder model. The diversification stage uses a mixture of experts to sample different binary masks on the source sequence for diverse content selection. The generation stage uses a standard encoder-decoder model given each selected content from the source sequence. Due to the non-differentiable nature of discrete sampling and the lack of ground truth labels for binary mask, we leverage a proxy for ground truth mask and adopt stochastic hard-EM for training. In question generation (SQuAD) and abstractive summarization (CNN-DM), our method demonstrates significant improvements in accuracy, diversity and training efficiency, including state-of-the-art top-1 accuracy in both datasets, 6% gain in top-5 accuracy, and 3.7 times faster training over a state of the art model. Our code is publicly available at https://github.com/clovaai/FocusSeq2Seq.