# Neural Predicting Higher-order Patterns in Temporal Networks

arXiv: https://arxiv.org/abs/2106.06039 · HTML: https://mlsanity.com/paper/2106.06039/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 62 references that are in the map.
- First posted on arXiv 10 Jun 2021.

## Builds on (most cited first, 27 of 27)

- [DeepWalk: Online Learning of Social Representations](https://mlsanity.com/paper/1403.6652/) (2014)
- [Link Prediction Based on Graph Neural Networks](https://mlsanity.com/paper/1802.09691/) (2018)
- [Temporal Graph Networks for Deep Learning on Dynamic Graphs](https://mlsanity.com/paper/2006.10637/) (2020)
- [Hypergraph Neural Networks](https://mlsanity.com/paper/1809.09401/) (2018)
- [Inductive Representation Learning on Temporal Graphs](https://mlsanity.com/paper/2002.07962/) (2020)
- [Representation Learning on Graphs: Methods and Applications](https://mlsanity.com/paper/1709.05584/) (2017)
- [Predicting Dynamic Embedding Trajectory in Temporal Interaction Networks](https://mlsanity.com/paper/1908.01207/) (2019)
- [EvolveGCN: Evolving Graph Convolutional Networks for Dynamic Graphs](https://mlsanity.com/paper/1902.10191/) (2019)
- [Time2Vec: Learning a Vector Representation of Time](https://mlsanity.com/paper/1907.05321/) (2019)
- [Labeling Trick: A Theory of Using Graph Neural Networks for Multi-Node Representation Learning](https://mlsanity.com/paper/2010.16103/) (2020)
- [Higher-order organization of complex networks](https://mlsanity.com/paper/1612.08447/) (2016)
- [Motifs in Temporal Networks](https://mlsanity.com/paper/1612.09259/) (2016)
- [Simplicial Closure and higher-order link prediction](https://mlsanity.com/paper/1802.06916/) (2018)
- [dyngraph2vec: Capturing Network Dynamics using Dynamic Graph Representation Learning](https://mlsanity.com/paper/1809.02657/) (2018)
- [Dynamic Graph Convolutional Networks](https://mlsanity.com/paper/1704.06199/) (2017)
- [Subgraph Neural Networks](https://mlsanity.com/paper/2006.10538/) (2020)
- [Hyper-SAGNN: a self-attention based graph neural network for hypergraphs](https://mlsanity.com/paper/1911.02613/) (2019)
- [Self-attention with Functional Time Representation Learning](https://mlsanity.com/paper/1911.12864/) (2019)
- [Inhomogeneous Hypergraph Clustering with Applications](https://mlsanity.com/paper/1709.01249/) (2017)
- [How Much and When Do We Need Higher-order Information in Hypergraphs? A Case Study on Hyperedge Prediction](https://mlsanity.com/paper/2001.11181/) (2020)
- [Structural Patterns and Generative Models of Real-world Hypergraphs](https://mlsanity.com/paper/2006.07060/) (2020)
- [Scalable Link Prediction in Dynamic Networks via Non-Negative Matrix Factorization](https://mlsanity.com/paper/1411.3675/) (2014)
- [Link Prediction via Higher-Order Motif Features](https://mlsanity.com/paper/1902.06679/) (2019)
- [Higher-order Network Analysis Takes Off, Fueled by Classical Ideas and New Data](https://mlsanity.com/paper/2103.05031/) (2021)
- [HONE: Higher-Order Network Embeddings](https://mlsanity.com/paper/1801.09303/) (2018)
- [Higher-Order Ranking and Link Prediction: From Closing Triangles to Closing Higher-Order Motifs](https://mlsanity.com/paper/1906.05059/) (2019)
- [Unsupervised Joint k-node Graph Representations with Compositional Energy-Based Models](https://mlsanity.com/paper/2010.04259/) (2020)

## Built on by (most cited first, 13 of 13)

- [Neighborhood-aware Scalable Temporal Network Representation Learning](https://mlsanity.com/paper/2209.01084/) (2022)
- [Equivariant Hypergraph Diffusion Neural Operators](https://mlsanity.com/paper/2207.06680/) (2022)
- [Algorithm and System Co-design for Efficient Subgraph-based Graph Representation Learning](https://mlsanity.com/paper/2202.13538/) (2022)
- [DyGMamba: Efficiently Modeling Long-Term Temporal Dependency on Continuous-Time Dynamic Graphs with State Space Models](https://mlsanity.com/paper/2408.04713/) (2024)
- [BenchTemp: A General Benchmark for Evaluating Temporal Graph Neural Networks](https://mlsanity.com/paper/2308.16385/) (2023)
- [SUREL+: Moving from Walks to Sets for Scalable Subgraph-based Graph Representation Learning](https://mlsanity.com/paper/2303.03379/) (2023)
- [CAT-Walk: Inductive Hypergraph Learning via Set Walks](https://mlsanity.com/paper/2306.11147/) (2023)
- [Temporal Graph Representation Learning with Adaptive Augmentation Contrastive](https://mlsanity.com/paper/2311.03897/) (2023)
- [Improving Graph Neural Networks on Multi-node Tasks with the Labeling Trick](https://mlsanity.com/paper/2304.10074/) (2023)
- [Scalable and Efficient Temporal Graph Representation Learning via Forward Recent Sampling](https://mlsanity.com/paper/2402.01964/) (2024)
- [Self-Exploring Language Models for Explainable Link Forecasting on Temporal Graphs via Reinforcement Learning](https://mlsanity.com/paper/2509.00975/) (2025)
- [Learning Short-Term and Long-Term Patterns of High-Order Dynamics in Real-World Networks](https://mlsanity.com/paper/2508.17236/) (2025)
- [Temporally Detailed Hypergraph Neural ODEs for Disease Progression Modeling](https://mlsanity.com/paper/2510.17211/) (2025)

## Abstract (arXiv)

Dynamic systems that consist of a set of interacting elements can be abstracted as temporal networks. Recently, higher-order patterns that involve multiple interacting nodes have been found crucial to indicate domain-specific laws of different temporal networks. This posts us the challenge of designing more sophisticated hypergraph models for these higher-order patterns and the associated new learning algorithms. Here, we propose the first model, named HIT, for full-spectrum higher-order pattern prediction in temporal hypergraphs. Particularly, we focus on predicting three types of common but important interaction patterns involving three interacting elements in temporal networks, which could be extended to even higher-order patterns. HIT extracts the structural representation of a node triplet of interest on the temporal hypergraph and uses it to tell what type of, when, and why the interaction expansion could happen in this triplet. HIT could achieve significant improvement (averaged 20% AUC gain to identify the interaction type, uniformly more accurate time estimation) compared to both heuristic and other neural-network-based baselines on 5 real-world large temporal hypergraphs. Moreover, HIT provides a certain degree of interpretability by identifying the most discriminatory structural features on the temporal hypergraphs for predicting different higher-order patterns.