# Set Interdependence Transformer: Set-to-Sequence Neural Networks for Permutation Learning and Structure Prediction

arXiv: https://arxiv.org/abs/2206.03720 · HTML: https://mlsanity.com/paper/2206.03720/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 37 references that are in the map.
- First posted on arXiv 8 Jun 2022.

## Builds on (most cited first, 12 of 12)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [Attention, Learn to Solve Routing Problems!](https://mlsanity.com/paper/1803.08475/) (2018)
- [A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Stories](https://mlsanity.com/paper/1604.01696/) (2016)
- [Order Matters: Sequence to sequence for sets](https://mlsanity.com/paper/1511.06391/) (2015)
- [RPM-Net: Robust Point Matching using Learned Features](https://mlsanity.com/paper/2003.13479/) (2020)
- [Learning a Deep Listwise Context Model for Ranking Refinement](https://mlsanity.com/paper/1804.05936/) (2018)
- [Robust Attentional Aggregation of Deep Feature Sets for Multi-view 3D Reconstruction](https://mlsanity.com/paper/1808.00758/) (2018)
- [DeepPermNet: Visual Permutation Learning](https://mlsanity.com/paper/1704.02729/) (2017)
- [Rep the Set: Neural Networks for Learning Set Representations](https://mlsanity.com/paper/1904.01962/) (2019)
- [Learning Representations of Sets through Optimized Permutations](https://mlsanity.com/paper/1812.03928/) (2018)
- [Set-to-Sequence Methods in Machine Learning: a Review](https://mlsanity.com/paper/2103.09656/) (2021)

## Built on by (most cited first, 2 of 2)

- [Advances in Set Function Learning: A Survey of Techniques and Applications](https://mlsanity.com/paper/2501.14991/) (2025)
- [Set2Seq Transformer: Temporal and Position-Aware Set Representations for Sequential Multiple-Instance Learning](https://mlsanity.com/paper/2408.03404/) (2024)

## Abstract (arXiv)

The task of learning to map an input set onto a permuted sequence of its elements is challenging for neural networks. Set-to-sequence problems occur in natural language processing, computer vision and structure prediction, where interactions between elements of large sets define the optimal output. Models must exhibit relational reasoning, handle varying cardinalities and manage combinatorial complexity. Previous attention-based methods require n layers of their set transformations to explicitly represent n-th order relations. Our aim is to enhance their ability to efficiently model higher-order interactions through an additional interdependence component. We propose a novel neural set encoding method called the Set Interdependence Transformer, capable of relating the set's permutation invariant representation to its elements within sets of any cardinality. We combine it with a permutation learning module into a complete, 3-part set-to-sequence model and demonstrate its state-of-the-art performance on a number of tasks. These range from combinatorial optimization problems, through permutation learning challenges on both synthetic and established NLP datasets for sentence ordering, to a novel domain of product catalog structure prediction. Additionally, the network's ability to generalize to unseen sequence lengths is investigated and a comparative empirical analysis of the existing methods' ability to learn higher-order interactions is provided.