mlsanityData to 7 Oct 2026

Set-to-Sequence Methods in Machine Learning: a Review

arXiv 2103.09656 ↗First posted 17 Mar 2021cs.LG cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

Machine learning on sets towards sequential output is an important and ubiquitous task, with applications ranging from language modeling and meta-learning to multi-agent strategy games and power grid optimization. Combining elements of representation learning and structured prediction, its two primary challenges include obtaining a meaningful, permutation invariant set representation and subsequently utilizing this representation to output a complex target permutation. This paper provides a comprehensive introduction to the field as well as an overview of important machine learning methods tackling both of these key challenges, with a detailed qualitative comparison of selected model architectures.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 93 of its 151 references that are in the map. First posted on arXiv 17 Mar 2021.

Builds on

93 of 151 references.

References · 93

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2017Graph Attention Networkscited by 8,854
  6. 2017Inductive Representation Learning on Large Graphscited by 6,284
  7. 2016Layer Normalizationcited by 5,579
  8. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  9. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  10. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  11. 2018How Powerful are Graph Neural Networks?cited by 4,355
  12. 2014Deep Learning Face Attributes in the Wildcited by 3,679
38 more
  1. 2016Pointer Sentinel Mixture Modelscited by 2,992
  2. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  3. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  4. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  5. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
  6. 2017Deep Setscited by 1,394
  7. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  8. 2019Attention is not Explanationcited by 916
  9. 2015Pointer Networkscited by 882
  10. 2020Object-Centric Learning with Slot Attentioncited by 865
  11. 2018Hierarchical Graph Representation Learning with Differentiable Poolingcited by 786
  12. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
  13. 2018Attention, Learn to Solve Routing Problems!cited by 646
  14. 2014Neural Turing Machinescited by 630
  15. 2018Universal Transformerscited by 613
  16. 2018Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizoncited by 612
  17. 2016Neural Combinatorial Optimization with Reinforcement Learningcited by 600
  18. 2017Learning Combinatorial Optimization Algorithms over Graphscited by 557
  19. 2018Object Detection with Deep Learning: A Reviewcited by 524
  20. 2016Adaptive Computation Time for Recurrent Neural Networkscited by 479
  21. 2021Dynamic Neural Networks: A Surveycited by 405
  22. 2018MolGAN: An implicit generative model for small molecular graphscited by 392
  23. 2014Memory Networkscited by 354
  24. 2018Reinforcement Learning for Solving the Vehicle Routing Problemcited by 327
  25. 2016Learning Convolutional Neural Networks for Graphscited by 314
  26. 2019Theoretical Limitations of Self-Attention in Neural Sequence Modelscited by 286
  27. 2018Actor-Attention-Critic for Multi-Agent Reinforcement Learningcited by 259
  28. 2015Order Matters: Sequence to sequence for setscited by 236
  29. 2020Fast Differentiable Sorting and Rankingcited by 196
  30. 2014Learning Fine-grained Image Similarity with Deep Rankingcited by 189
  31. 2019TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agentscited by 188
  32. 2018Learning Latent Permutations with Gumbel-Sinkhorn Networkscited by 172
  33. 2016Long Short-Term Memory-Networks for Machine Readingcited by 161
  34. 2016An Actor-Critic Algorithm for Sequence Predictioncited by 158
  35. 2020On Translation Invariance in CNNs: Convolutional Layers can Exploit Absolute Spatial Locationcited by 141
  36. 2016Sequence-to-Sequence Learning as Beam-Search Optimizationcited by 126
  37. 2019An Attentive Survey of Attention Modelscited by 123
  38. 2017CondenseNet: An Efficient DenseNet using Learned Group Convolutionscited by 121

50 most cited of 93 All 93 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 2; 2023: 0; 2024: 2; 2025: 0; 2026: 1.

6 papers.

  1. 2021Learning Prototype-oriented Set Representations for Meta-Learningcited by 19
  2. 2026The Taxonomies, Training, and Applications of Event Stream Modelling for Electronic Health Recordscited by 3
  3. 2022Set Interdependence Transformer: Set-to-Sequence Neural Networks for Permutation Learning and Structure Predictioncited by 2
  4. 2022Towards Improving the Generation Quality of Autoregressive Slot VAEscited by 1
  5. 2024Versatile Ordering Network: An Attention-based Neural Network for Ordering Across Scales and Quality Metricscited by 1
  6. 2024Set2Seq Transformer: Temporal and Position-Aware Set Representations for Sequential Multiple-Instance Learningcited by 0