mlsanityData to 14 Oct 2026

Conditional set generation using Seq2seq models

arXiv 2205.12485 ↗First posted 25 May 2022cs.CL cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

Conditional set generation learns a mapping from an input sequence of tokens to a set. Several NLP tasks, such as entity typing and dialogue emotion tagging, are instances of set generation. Seq2Seq models, a popular choice for set generation, treat a set as a sequence and do not fully leverage its key properties, namely order-invariance and cardinality. We propose a novel algorithm for effectively sampling informative orders over the combinatorial space of label orders. We jointly model the set cardinality and output by prepending the set size and taking advantage of the autoregressive factorization used by Seq2Seq models. Our method is a model-independent data augmentation approach that endows any Seq2Seq model with the signals of order-invariance and cardinality. Training a Seq2Seq model on this augmented data (without any additional annotations) gets an average relative improvement of 20% on four benchmark datasets across various models: BART, T5, and GPT-3. Code to use SETAUG available at: https://setgen.structgen.com.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 42 of its 60 references that are in the map. First posted on arXiv 25 May 2022.

Builds on

42 of 60 references.

References · 42

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  5. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  6. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  7. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  8. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  9. 2017Deep Setscited by 1,421
  10. 2018Hierarchical Neural Story Generationcited by 1,193
  11. 2018Junction Tree Variational Autoencoder for Molecular Graph Generationcited by 652
  12. 2019ERNIE: Enhanced Language Representation with Informative Entitiescited by 528
30 more
  1. 2019Neural Text Generation with Unlikelihood Trainingcited by 503
  2. 2020GoEmotions: A Dataset of Fine-Grained Emotionscited by 486
  3. 2015Importance Weighted Autoencoderscited by 456
  4. 2017A simple neural network module for relational reasoningcited by 346
  5. 2020K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapterscited by 319
  6. 2020COMET-ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphscited by 254
  7. 2015Order Matters: Sequence to sequence for setscited by 237
  8. 2018Sharp Nearby, Fuzzy Far Away: How Neural Language Models Use Contextcited by 113
  9. 2016Recurrent Neural Network Grammarscited by 109
  10. 2017Deep Keyphrase Generationcited by 102
  11. 2018Janossy Pooling: Learning Deep Permutation-Invariant Functions for Variable-Size Inputscited by 98
  12. 2018SGM: Sequence Generation Model for Multi-label Classificationcited by 89
  13. 2018Ultra-Fine Entity Typingcited by 87
  14. 2020A Theoretical Analysis of the Repetition Problem in Text Generationcited by 83
  15. 2021SetVAE: Learning Hierarchical Composition for Generative Modeling of Set-Structured Datacited by 78
  16. 2019Deep Set Prediction Networkscited by 71
  17. 2018Importance Weighting and Variational Inferencecited by 55
  18. 2018One Size Does Not Fit All: Generating and Evaluating Variable Number of Keyphrasescited by 52
  19. 2020Generative Data Augmentation for Commonsense Reasoningcited by 49
  20. 2021One2Set: Generating Diverse Keyphrases as a Setcited by 41
  21. 2021Ultra-Fine Entity Typing with Weak Supervision from a Masked Language Modelcited by 39
  22. 2021Order Matters: Probabilistic Modeling of Node Sequence for Graph Generationcited by 33
  23. 2020Conditional Set Generation with Transformerscited by 32
  24. 2020Neural Learning of One-of-Many Solutions for Combinatorial Problems in Structured Output Spacescited by 14
  25. 2020Set Prediction without Imposing Structure as Conditional Density Estimationcited by 12
  26. 2020Learn to Predict Sets Using Feed-Forward Neural Networkscited by 10
  27. 2019Does Order Matter? An Empirical Study on Generating Multiple Keyphrases as a Sequencecited by 8
  28. 2019Adapting RNN Sequence Prediction Model to Multi-label Set Predictioncited by 7
  29. 2018Deep Perm-Set Net: Learn to predict sets with unknown permutation and cardinality using deep neural networkscited by 6
  30. 2019Know2Look: Commonsense Knowledge for Visual Searchcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 3; 2025: 2; 2026: 1.

7 papers.

  1. 2024To Code, or Not To Code? Exploring Impact of Code in Pre-trainingcited by 47
  2. 2023A Unified Generative Approach to Product Attribute-Value Identificationcited by 10
  3. 2024Set-Aligning Framework for Auto-Regressive Event Temporal Graph Generationcited by 8
  4. 2024Exploring Key Point Analysis with Pairwise Generation and Graph Partitioningcited by 4
  5. 2025Zero-Shot Keyphrase Generation: Investigating Specialized Instructions and Multi-Sample Aggregation on Large Language Modelscited by 0
  6. 2025Post-Training Language Models for Continual Relation Extractioncited by 0
  7. 2026Beyond Triplet Plausibility: Relation Set Completion in Knowledge Graphscited by 0