# Continuous-Time Sequential Recommendation with Temporal Graph Collaborative Transformer

arXiv: https://arxiv.org/abs/2108.06625 · HTML: https://mlsanity.com/paper/2108.06625/

## Key facts

- Cited by 69 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 57 references that are in the map.
- First posted on arXiv 14 Aug 2021.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation](https://mlsanity.com/paper/2002.02126/) (2020)
- [Neural Collaborative Filtering](https://mlsanity.com/paper/1708.05031/) (2017)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Image-based Recommendations on Styles and Substitutes](https://mlsanity.com/paper/1506.04757/) (2015)
- [Neural Graph Collaborative Filtering](https://mlsanity.com/paper/1905.08108/) (2019)
- [Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding](https://mlsanity.com/paper/1809.07426/) (2018)
- [Inductive Representation Learning on Temporal Graphs](https://mlsanity.com/paper/2002.07962/) (2020)
- [Predicting Dynamic Embedding Trajectory in Temporal Interaction Networks](https://mlsanity.com/paper/1908.01207/) (2019)
- [KGAT: Knowledge Graph Attention Network for Recommendation](https://mlsanity.com/paper/1905.07854/) (2019)
- [Session-based Recommendation with Graph Neural Networks](https://mlsanity.com/paper/1811.00855/) (2018)
- [Neural Attentive Session-based Recommendation](https://mlsanity.com/paper/1711.04725/) (2017)
- [Personalizing Session-based Recommendations with Hierarchical Recurrent Neural Networks](https://mlsanity.com/paper/1706.04148/) (2017)
- [Modeling User Exposure in Recommendation](https://mlsanity.com/paper/1510.07025/) (2015)
- [Self-attention with Functional Time Representation Learning](https://mlsanity.com/paper/1911.12864/) (2019)
- [Augmenting Sequential Recommendation with Pseudo-Prior Items via Reversely Pre-training Transformer](https://mlsanity.com/paper/2105.00522/) (2021)
- [Dynamic Graph Collaborative Filtering](https://mlsanity.com/paper/2101.02844/) (2021)
- [Are Pretrained Transformers Robust in Intent Classification? A Missing Ingredient in Evaluation of Out-of-Scope Intent Detection](https://mlsanity.com/paper/2106.04564/) (2021)
- [Modeling Sequences as Distributions with Uncertainty for Sequential Recommendation](https://mlsanity.com/paper/2106.06165/) (2021)
- [BasConv: Aggregating Heterogeneous Interactions for Basket Recommendation with Graph Convolutional Neural Network](https://mlsanity.com/paper/2001.09900/) (2020)
- [Basket Recommendation with Multi-Intent Translation Graph Neural Network](https://mlsanity.com/paper/2010.11419/) (2020)
- [D'ej`a vu: A Contextualized Temporal Attention Mechanism for Sequential Recommendation](https://mlsanity.com/paper/2002.00741/) (2020)
- [Enriching Non-Autoregressive Transformer with Syntactic and SemanticStructures for Neural Machine Translation](https://mlsanity.com/paper/2101.08942/) (2021)
- [JSCN: Joint Spectral Convolutional Network for Cross Domain Recommendation](https://mlsanity.com/paper/1910.08219/) (2019)
- [M2: Mixed Models with Preferences, Popularities and Transitions for Next-Basket Recommendation](https://mlsanity.com/paper/2004.01646/) (2020)

## Built on by (most cited first, 30 of 69)

- [Towards Better Dynamic Graph Learning: New Architecture and Unified Library](https://mlsanity.com/paper/2303.13047/) (2023)
- [Do We Really Need Complicated Model Architectures For Temporal Networks?](https://mlsanity.com/paper/2302.11636/) (2023)
- [DiffuRec: A Diffusion Model for Sequential Recommendation](https://mlsanity.com/paper/2304.00686/) (2023)
- [Intent Contrastive Learning for Sequential Recommendation](https://mlsanity.com/paper/2202.02519/) (2022)
- [A Comprehensive Survey on Deep Graph Representation Learning](https://mlsanity.com/paper/2304.05055/) (2023)
- [EasyTPP: Towards Open Benchmarking Temporal Point Processes](https://mlsanity.com/paper/2307.08097/) (2023)
- [Sequential Recommendation via Stochastic Self-Attention](https://mlsanity.com/paper/2201.06035/) (2022)
- [A Systematic Review and Replicability Study of BERT4Rec for Sequential Recommendation](https://mlsanity.com/paper/2207.07483/) (2022)
- [Improving Temporal Link Prediction via Temporal Walk Matrix Projection](https://mlsanity.com/paper/2410.04013/) (2024)
- [DSKReG: Differentiable Sampling on Knowledge Graph for Recommendation with Relational GNN](https://mlsanity.com/paper/2108.11883/) (2021)
- [Hierarchical Time-Aware Mixture of Experts for Multi-Modal Sequential Recommendation](https://mlsanity.com/paper/2501.14269/) (2025)
- [HETFORMER: Heterogeneous Transformer with Sparse Attention for Long-Text Extractive Summarization](https://mlsanity.com/paper/2110.06388/) (2021)
- [SelfGNN: Self-Supervised Graph Neural Networks for Sequential Recommendation](https://mlsanity.com/paper/2405.20878/) (2024)
- [TransGNN: Harnessing the Collaborative Power of Transformers and Graph Neural Networks for Recommender Systems](https://mlsanity.com/paper/2308.14355/) (2023)
- [Graph Masked Autoencoders with Transformers](https://mlsanity.com/paper/2202.08391/) (2022)
- [HOT: Higher-Order Dynamic Graph Representation Learning with Efficient Transformers](https://mlsanity.com/paper/2311.18526/) (2023)
- [End-to-end Learnable Clustering for Intent Learning in Recommendation](https://mlsanity.com/paper/2401.05975/) (2024)
- [Co-Neighbor Encoding Schema: A Light-cost Structure Encoding Method for Dynamic Link Prediction](https://mlsanity.com/paper/2407.20871/) (2024)
- [DyGKT: Dynamic Graph Learning for Knowledge Tracing](https://mlsanity.com/paper/2407.20824/) (2024)
- [Continuous-Time User Preference Modelling for Temporal Sets Prediction](https://mlsanity.com/paper/2204.05490/) (2022)
- [EasyDGL: Encode, Train and Interpret for Continuous-time Dynamic Graph Learning](https://mlsanity.com/paper/2303.12341/) (2023)
- [Time Interval-enhanced Graph Neural Network for Shared-account Cross-domain Sequential Recommendation](https://mlsanity.com/paper/2206.08050/) (2022)
- [Dynamic Graph Transformer with Correlated Spatial-Temporal Positional Encoding](https://mlsanity.com/paper/2407.16959/) (2024)
- [Modeling Temporal Positive and Negative Excitation for Sequential Recommendation](https://mlsanity.com/paper/2410.22013/) (2024)
- [Unifying Text Semantics and Graph Structures for Temporal Text-attributed Graphs with Large Language Models](https://mlsanity.com/paper/2503.14411/) (2025)
- [Improving Contrastive Learning with Model Augmentation](https://mlsanity.com/paper/2203.15508/) (2022)
- [Exploiting Session Information in BERT-based Session-aware Sequential Recommendation](https://mlsanity.com/paper/2204.10851/) (2022)
- [Long Short-Term Preference Modeling for Continuous-Time Sequential Recommendation](https://mlsanity.com/paper/2208.00593/) (2022)
- [SINCERE: Sequential Interaction Networks representation learning on Co-Evolving RiEmannian manifolds](https://mlsanity.com/paper/2305.03883/) (2023)
- [Integrating Item Relevance in Training Loss for Sequential Recommender Systems](https://mlsanity.com/paper/2305.10824/) (2023)

## Abstract (arXiv)

In order to model the evolution of user preference, we should learn user/item embeddings based on time-ordered item purchasing sequences, which is defined as Sequential Recommendation (SR) problem. Existing methods leverage sequential patterns to model item transitions. However, most of them ignore crucial temporal collaborative signals, which are latent in evolving user-item interactions and coexist with sequential patterns. Therefore, we propose to unify sequential patterns and temporal collaborative signals to improve the quality of recommendation, which is rather challenging. Firstly, it is hard to simultaneously encode sequential patterns and collaborative signals. Secondly, it is non-trivial to express the temporal effects of collaborative signals. Hence, we design a new framework Temporal Graph Sequential Recommender (TGSRec) upon our defined continuous-time bi-partite graph. We propose a novel Temporal Collaborative Trans-former (TCT) layer in TGSRec, which advances the self-attention mechanism by adopting a novel collaborative attention. TCT layer can simultaneously capture collaborative signals from both users and items, as well as considering temporal dynamics inside sequential patterns. We propagate the information learned fromTCTlayerover the temporal graph to unify sequential patterns and temporal collaborative signals. Empirical results on five datasets show that TGSRec significantly outperforms other baselines, in average up to 22.5% and 22.1%absolute improvements in Recall@10and MRR, respectively.