# The CAT SET on the MAT: Cross Attention for Set Matching in Bipartite Hypergraphs

arXiv: https://arxiv.org/abs/2111.00243 · HTML: https://mlsanity.com/paper/2111.00243/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 52 references that are in the map.
- First posted on arXiv 30 Oct 2021.

## Builds on (most cited first, 20 of 20)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Hypergraph Neural Networks](https://mlsanity.com/paper/1809.09401/) (2018)
- [Order Matters: Sequence to sequence for sets](https://mlsanity.com/paper/1511.06391/) (2015)
- [Hypergraph Convolution and Hypergraph Attention](https://mlsanity.com/paper/1901.08150/) (2019)
- [Simplicial Closure and higher-order link prediction](https://mlsanity.com/paper/1802.06916/) (2018)
- [On the Limitations of Representing Functions on Sets](https://mlsanity.com/paper/1901.09006/) (2019)
- [Hyper-SAGNN: a self-attention based graph neural network for hypergraphs](https://mlsanity.com/paper/1911.02613/) (2019)
- [FSPool: Learning Set Representations with Featurewise Sort Pooling](https://mlsanity.com/paper/1906.02795/) (2019)
- [Structural Deep Embedding for Hyper-Networks](https://mlsanity.com/paper/1711.10146/) (2017)
- [Line Hypergraph Convolution Network: Applying Graph Convolution for Hypergraphs](https://mlsanity.com/paper/2002.03392/) (2020)
- [Deep Hyperedges: a Framework for Transductive and Inductive Learning on Hypergraphs](https://mlsanity.com/paper/1910.02633/) (2019)
- [Powerset Convolutional Neural Networks](https://mlsanity.com/paper/1909.02253/) (2019)
- [Finding Perfect Matchings in Bipartite Hypergraphs](https://mlsanity.com/paper/1509.07007/) (2015)
- [Latent geometry of bipartite networks](https://mlsanity.com/paper/1610.09048/) (2016)
- [Exchangeable deep neural networks for set-to-set matching and learning](https://mlsanity.com/paper/1910.09972/) (2019)

## Built on by (most cited first, 1 of 1)

- [Dynamic Representation Learning with Temporal Point Processes for Higher-Order Interaction Forecasting](https://mlsanity.com/paper/2112.10154/) (2021)

## Abstract (arXiv)

Usual relations between entities could be captured using graphs; but those of a higher-order -- more so between two different types of entities (which we term "left" and "right") -- calls for a "bipartite hypergraph". For example, given a left set of symptoms and right set of diseases, the relation between a set subset of symptoms (that a patient experiences at a given point of time) and a subset of diseases (that he/she might be diagnosed with) could be well-represented using a bipartite hyperedge. The state-of-the-art in embedding nodes of a hypergraph is based on learning the self-attention structure between node-pairs from a hyperedge. In the present work, given a bipartite hypergraph, we aim at capturing relations between node pairs from the cross-product between the left and right hyperedges, and term it a "cross-attention" (CAT) based model. More precisely, we pose "bipartite hyperedge link prediction" as a set-matching (SETMAT) problem and propose a novel neural network architecture called CATSETMAT for the same. We perform extensive experiments on multiple bipartite hypergraph datasets to show the superior performance of CATSETMAT, which we compare with multiple techniques from the state-of-the-art. Our results also elucidate information flow in self- and cross-attention scenarios.