# Inductive Representation Learning in Temporal Networks via Causal Anonymous Walks

arXiv: https://arxiv.org/abs/2101.05974 · HTML: https://mlsanity.com/paper/2101.05974/

## Key facts

- Cited by 193 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 60 references that are in the map.
- First posted on arXiv 15 Jan 2021.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Variational Graph Auto-Encoders](https://mlsanity.com/paper/1611.07308/) (2016)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [Temporal Graph Networks for Deep Learning on Dynamic Graphs](https://mlsanity.com/paper/2006.10637/) (2020)
- [Inductive Representation Learning on Temporal Graphs](https://mlsanity.com/paper/2002.07962/) (2020)
- [Representation Learning on Graphs: Methods and Applications](https://mlsanity.com/paper/1709.05584/) (2017)
- [Predicting Dynamic Embedding Trajectory in Temporal Interaction Networks](https://mlsanity.com/paper/1908.01207/) (2019)
- [EvolveGCN: Evolving Graph Convolutional Networks for Dynamic Graphs](https://mlsanity.com/paper/1902.10191/) (2019)
- [Methods for Interpreting and Understanding Deep Neural Networks](https://mlsanity.com/paper/1706.07979/) (2017)
- [Time2Vec: Learning a Vector Representation of Time](https://mlsanity.com/paper/1907.05321/) (2019)
- [Know-Evolve: Deep Temporal Reasoning for Dynamic Knowledge Graphs](https://mlsanity.com/paper/1705.05742/) (2017)
- [Higher-order organization of complex networks](https://mlsanity.com/paper/1612.08447/) (2016)
- [Motifs in Temporal Networks](https://mlsanity.com/paper/1612.09259/) (2016)
- [Simplicial Closure and higher-order link prediction](https://mlsanity.com/paper/1802.06916/) (2018)
- [dyngraph2vec: Capturing Network Dynamics using Dynamic Graph Representation Learning](https://mlsanity.com/paper/1809.02657/) (2018)
- [Variational Graph Recurrent Neural Networks](https://mlsanity.com/paper/1908.09710/) (2019)
- [Dynamic Graph Convolutional Networks](https://mlsanity.com/paper/1704.06199/) (2017)
- [Self-attention with Functional Time Representation Learning](https://mlsanity.com/paper/1911.12864/) (2019)
- [Anonymous Walk Embeddings](https://mlsanity.com/paper/1805.11921/) (2018)
- [Node Embedding over Temporal Graphs](https://mlsanity.com/paper/1903.08889/) (2019)
- [dynnode2vec: Scalable Dynamic Network Embedding](https://mlsanity.com/paper/1812.02356/) (2018)
- [Inhomogeneous Hypergraph Clustering with Applications](https://mlsanity.com/paper/1709.01249/) (2017)
- [E-LSTM-D: A Deep Learning Framework for Dynamic Network Link Prediction](https://mlsanity.com/paper/1902.08329/) (2019)
- [Link Prediction via Higher-Order Motif Features](https://mlsanity.com/paper/1902.06679/) (2019)
- [Temporal motifs reveal homophily, gender-specific patterns and group talk in mobile communication networks](https://mlsanity.com/paper/1302.2563/) (2013)
- [Higher-Order Ranking and Link Prediction: From Closing Triangles to Closing Higher-Order Motifs](https://mlsanity.com/paper/1906.05059/) (2019)

## Built on by (most cited first, 30 of 193)

- [Towards Better Dynamic Graph Learning: New Architecture and Unified Library](https://mlsanity.com/paper/2303.13047/) (2023)
- [Do We Really Need Complicated Model Architectures For Temporal Networks?](https://mlsanity.com/paper/2302.11636/) (2023)
- [Temporal Graph Benchmark for Machine Learning on Temporal Graphs](https://mlsanity.com/paper/2307.01026/) (2023)
- [Towards Better Evaluation for Dynamic Link Prediction](https://mlsanity.com/paper/2207.10128/) (2022)
- [Equivariant and Stable Positional Encoding for More Powerful Graph Neural Networks](https://mlsanity.com/paper/2203.00199/) (2022)
- [A Survey on Graph Representation Learning Methods](https://mlsanity.com/paper/2204.01855/) (2022)
- [TGL: A General Framework for Temporal GNN Training on Billion-Scale Graphs](https://mlsanity.com/paper/2203.14883/) (2022)
- [Graph Neural Networks for temporal graphs: State of the art, open challenges, and opportunities](https://mlsanity.com/paper/2302.01018/) (2023)
- [Provably expressive temporal graph networks](https://mlsanity.com/paper/2209.15059/) (2022)
- [Neighborhood-aware Scalable Temporal Network Representation Learning](https://mlsanity.com/paper/2209.01084/) (2022)
- [Walking Out of the Weisfeiler Leman Hierarchy: Graph Learning Beyond Message Passing](https://mlsanity.com/paper/2102.08786/) (2021)
- [Structural Re-weighting Improves Graph Domain Adaptation](https://mlsanity.com/paper/2306.03221/) (2023)
- [LLM4DyG: Can Large Language Models Solve Spatial-Temporal Problems on Dynamic Graphs?](https://mlsanity.com/paper/2310.17110/) (2023)
- [OOD Link Prediction Generalization Capabilities of Message-Passing GNNs in Larger Test Graphs](https://mlsanity.com/paper/2205.15117/) (2022)
- [A Comprehensive Survey of Dynamic Graph Neural Networks: Models, Frameworks, Benchmarks, Experiments and Challenges](https://mlsanity.com/paper/2405.00476/) (2024)
- [Algorithm and System Co-design for Efficient Subgraph-based Graph Representation Learning](https://mlsanity.com/paper/2202.13538/) (2022)
- [Deep Temporal Graph Clustering](https://mlsanity.com/paper/2305.10738/) (2023)
- [TempME: Towards the Explainability of Temporal Graph Neural Networks via Motif Discovery](https://mlsanity.com/paper/2310.19324/) (2023)
- [DTGB: A Comprehensive Benchmark for Dynamic Text-Attributed Graphs](https://mlsanity.com/paper/2406.12072/) (2024)
- [Environment-Aware Dynamic Graph Learning for Out-of-Distribution Generalization](https://mlsanity.com/paper/2311.11114/) (2023)
- [Scaling Up Dynamic Graph Representation Learning via Spiking Neural Networks](https://mlsanity.com/paper/2208.10364/) (2022)
- [A Self-supervised Riemannian GNN with Time Varying Curvature for Temporal Graph Learning](https://mlsanity.com/paper/2208.14073/) (2022)
- [A survey of dynamic graph neural networks](https://mlsanity.com/paper/2404.18211/) (2024)
- [Relational Deep Learning: Challenges, Foundations and Next-Generation Architectures](https://mlsanity.com/paper/2506.16654/) (2025)
- [Improving Temporal Link Prediction via Temporal Walk Matrix Projection](https://mlsanity.com/paper/2410.04013/) (2024)
- [Inductive Representation Learning in Temporal Networks via Mining Neighborhood and Community Influences](https://mlsanity.com/paper/2110.00267/) (2021)
- [Deep learning for dynamic graphs: models and benchmarks](https://mlsanity.com/paper/2307.06104/) (2023)
- [Spectral Invariant Learning for Dynamic Graphs under Distribution Shifts](https://mlsanity.com/paper/2403.05026/) (2024)
- [Learning Long Range Dependencies on Graphs via Random Walks](https://mlsanity.com/paper/2406.03386/) (2024)
- [Efficient Neural Common Neighbor for Temporal Graph Link Prediction](https://mlsanity.com/paper/2406.07926/) (2024)

## Abstract (arXiv)

Temporal networks serve as abstractions of many real-world dynamic systems. These networks typically evolve according to certain laws, such as the law of triadic closure, which is universal in social networks. Inductive representation learning of temporal networks should be able to capture such laws and further be applied to systems that follow the same laws but have not been unseen during the training stage. Previous works in this area depend on either network node identities or rich edge attributes and typically fail to extract these laws. Here, we propose Causal Anonymous Walks (CAWs) to inductively represent a temporal network. CAWs are extracted by temporal random walks and work as automatic retrieval of temporal network motifs to represent network dynamics while avoiding the time-consuming selection and counting of those motifs. CAWs adopt a novel anonymization strategy that replaces node identities with the hitting counts of the nodes based on a set of sampled walks to keep the method inductive, and simultaneously establish the correlation between motifs. We further propose a neural-network model CAW-N to encode CAWs, and pair it with a CAW sampling strategy with constant memory and time cost to support online training and inference. CAW-N is evaluated to predict links over 6 real temporal networks and uniformly outperforms previous SOTA methods by averaged 10% AUC gain in the inductive setting. CAW-N also outperforms previous methods in 4 out of the 6 networks in the transductive setting.