# Adapting RNN Sequence Prediction Model to Multi-label Set Prediction

arXiv: https://arxiv.org/abs/1904.05829 · HTML: https://mlsanity.com/paper/1904.05829/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Apr 2019.

## Built on by (most cited first, 7 of 7)

- [Enhancing Label Correlation Feedback in Multi-Label Text Classification via Multi-Task Learning](https://mlsanity.com/paper/2106.03103/) (2021)
- [Set-Aligning Framework for Auto-Regressive Event Temporal Graph Generation](https://mlsanity.com/paper/2404.01532/) (2024)
- [Conditional set generation using Seq2seq models](https://mlsanity.com/paper/2205.12485/) (2022)
- [Extending Multi-Sense Word Embedding to Phrases and Sentences for Unsupervised Semantic Applications](https://mlsanity.com/paper/2103.15330/) (2021)
- [Advances in Set Function Learning: A Survey of Techniques and Applications](https://mlsanity.com/paper/2501.14991/) (2025)
- [Conversation Kernels: A Flexible Mechanism to Learn Relevant Context for Online Conversation Understanding](https://mlsanity.com/paper/2505.20482/) (2025)
- [Beyond Triplet Plausibility: Relation Set Completion in Knowledge Graphs](https://mlsanity.com/paper/2606.29860/) (2026)

## Abstract (arXiv)

We present an adaptation of RNN sequence models to the problem of multi-label classification for text, where the target is a set of labels, not a sequence. Previous such RNN models define probabilities for sequences but not for sets; attempts to obtain a set probability are after-thoughts of the network design, including pre-specifying the label order, or relating the sequence probability to the set probability in ad hoc ways. Our formulation is derived from a principled notion of set probability, as the sum of probabilities of corresponding permutation sequences for the set. We provide a new training objective that maximizes this set probability, and a new prediction objective that finds the most probable set on a test document. These new objectives are theoretically appealing because they give the RNN model freedom to discover the best label order, which often is the natural one (but different among documents). We develop efficient procedures to tackle the computation difficulties involved in training and prediction. Experiments on benchmark datasets demonstrate that we outperform state-of-the-art methods for this task.