# Semantics-enhanced Temporal Graph Networks for Content Popularity Prediction

arXiv: https://arxiv.org/abs/2301.12355 · HTML: https://mlsanity.com/paper/2301.12355/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 54 references that are in the map.
- First posted on arXiv 29 Jan 2023.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [Graph Neural Networks: A Review of Methods and Applications](https://mlsanity.com/paper/1812.08434/) (2018)
- [Image Transformer](https://mlsanity.com/paper/1802.05751/) (2018)
- [Temporal Graph Networks for Deep Learning on Dynamic Graphs](https://mlsanity.com/paper/2006.10637/) (2020)
- [Pre-trained Models for Natural Language Processing: A Survey](https://mlsanity.com/paper/2003.08271/) (2020)
- [Graph Neural Networks in Recommender Systems: A Survey](https://mlsanity.com/paper/2011.02260/) (2020)
- [Inductive Representation Learning on Temporal Graphs](https://mlsanity.com/paper/2002.07962/) (2020)
- [RippleNet: Propagating User Preferences on the Knowledge Graph for Recommender Systems](https://mlsanity.com/paper/1803.03467/) (2018)
- [Foundations and modelling of dynamic networks using Dynamic Graph Neural Networks: A survey](https://mlsanity.com/paper/2005.07496/) (2020)
- [Communication-Efficient Edge AI: Algorithms and Systems](https://mlsanity.com/paper/2002.09668/) (2020)
- [Learnable Fourier Features for Multi-Dimensional Spatial Positional Encoding](https://mlsanity.com/paper/2106.02795/) (2021)
- [Improving Question Answering by Commonsense-Based Pre-Training](https://mlsanity.com/paper/1809.03568/) (2018)
- [Wireless Deep Video Semantic Transmission](https://mlsanity.com/paper/2205.13129/) (2022)
- [Mobile Edge Intelligence and Computing for the Internet of Vehicles](https://mlsanity.com/paper/1906.00400/) (2019)
- [Contextualized Graph Attention Network for Recommendation with Item Knowledge Graph](https://mlsanity.com/paper/2004.11529/) (2020)
- [TEDGE-Caching: Transformer-based Edge Caching Towards 6G Networks](https://mlsanity.com/paper/2112.00633/) (2021)
- [Deep Learning for Wireless Coded Caching with Unknown and Time-Variant Content Popularity](https://mlsanity.com/paper/2008.09422/) (2020)
- [Cocktail Edge Caching: Ride Dynamic Trends of Content Popularity with Ensemble Learning](https://mlsanity.com/paper/2101.05885/) (2021)
- [AoI-based Temporal Attention Graph Neural Network for Popularity Prediction and Content Caching](https://mlsanity.com/paper/2208.08606/) (2022)

## Built on by (most cited first, 1 of 1)

- [Topology-Assisted Spatio-Temporal Pattern Disentangling for Scalable MARL in Large-scale Autonomous Traffic Control](https://mlsanity.com/paper/2506.12453/) (2025)

## Abstract (arXiv)

The surging demand for high-definition video streaming services and large neural network models (e.g., Generative Pre-trained Transformer, GPT) implies a tremendous explosion of Internet traffic. To mitigate the traffic pressure, architectures with in-network storage have been proposed to cache popular contents at devices in closer proximity to users. Correspondingly, in order to maximize caching utilization, it becomes essential to devise an effective popularity prediction method. In that regard, predicting popularity with dynamic graph neural network (DGNN) models achieve remarkable performance. However, DGNN models still suffer from tackling sparse datasets where most users are inactive. Therefore, we propose a reformative temporal graph network, named semantics-enhanced temporal graph network (STGN), which attaches extra semantic information into the user-content bipartite graph and could better leverage implicit relationships behind the superficial topology structure. On top of that, we customize its temporal and structural learning modules to further boost the prediction performance. Specifically, in order to efficiently aggregate the diversified semantics that a content might possess, we design a user-specific attention (UsAttn) mechanism for temporal learning module. Unlike the attention mechanism that only analyzes the influence of genres on content, UsAttn also considers the attraction of semantic information to a specific user. Meanwhile, as for the structural learning, we introduce the concept of positional encoding into our attention-based graph learning and adopt a semantic positional encoding (SPE) function to facilitate the analysis of content-oriented user-association analysis. Finally, extensive simulations verify the superiority of our STGN models and demonstrate the effectiveness in content caching.