mlsanityData to 14 Oct 2026

Unifying Text Semantics and Graph Structures for Temporal Text-attributed Graphs with Large Language Models

arXiv 2503.14411 ↗First posted 18 Mar 2025cs.CL cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

Temporal graph neural networks (TGNNs) have shown remarkable performance in temporal graph modeling. However, real-world temporal graphs often possess rich textual information, giving rise to temporal text-attributed graphs (TTAGs). Such combination of dynamic text semantics and evolving graph structures introduces heightened complexity. Existing TGNNs embed texts statically and rely heavily on encoding mechanisms that biasedly prioritize structural information, overlooking the temporal evolution of text semantics and the essential interplay between semantics and structures for synergistic reinforcement. To tackle these issues, we present CROSS, a flexible framework that seamlessly extends existing TGNNs for TTAG modeling. CROSS is designed by decomposing the TTAG modeling process into two phases: (i) temporal semantics extraction; and (ii) semantic-structural information unification. The key idea is to advance the large language models (LLMs) to dynamically extract the temporal semantics in text space and then generate cohesive representations unifying both semantics and structures. Specifically, we propose a Temporal Semantics Extractor in the CROSS framework, which empowers LLMs to offer the temporal semantic understanding of node's evolving contexts of textual neighborhoods, facilitating semantic dynamics. Subsequently, we introduce the Semantic-structural Co-encoder, which collaborates with the above Extractor for synthesizing illuminating representations by jointly considering both semantic and structural information while encouraging their mutual reinforcement. Extensive experiments show that CROSS achieves state-of-the-art results on four public datasets and one industrial dataset, with 24.7% absolute MRR gain on average in temporal link prediction and 3.7% AUC gain in node classification of industrial application.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 62 of its 84 references that are in the map. First posted on arXiv 18 Mar 2025.

Builds on

62 of 84 references.

References · 62

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023Mistral 7Bcited by 3,164
  3. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,463
  4. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
  5. 2020Temporal Graph Networks for Deep Learning on Dynamic Graphscited by 530
  6. 2020Inductive Representation Learning on Temporal Graphscited by 462
  7. 2019Predicting Dynamic Embedding Trajectory in Temporal Interaction Networkscited by 448
  8. 2019EvolveGCN: Evolving Graph Convolutional Networks for Dynamic Graphscited by 447
  9. 2023Exploring the Potential of Large Language Models (LLMs) in Learning on Graphscited by 274
  10. 2019Representation Learning for Dynamic Graphs: A Surveycited by 265
  11. 2023GraphGPT: Graph Instruction Tuning for Large Language Modelscited by 226
  12. 2024G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answeringcited by 223
38 more
  1. 2019Time2Vec: Learning a Vector Representation of Timecited by 213
  2. 2022LinkBERT: Pretraining Language Models with Document Linkscited by 211
  3. 2021Inductive Representation Learning in Temporal Networks via Causal Anonymous Walkscited by 193
  4. 2022GreaseLM: Graph REASoning Enhanced Language Models for Question Answeringcited by 171
  5. 2023Towards Better Dynamic Graph Learning: New Architecture and Unified Librarycited by 157
  6. 2022Learning on Large-scale Text-attributed Graphs via Variational Inferencecited by 155
  7. 2023Representation Learning with Large Language Models for Recommendationcited by 155
  8. 2021GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graphcited by 143
  9. 2023Do We Really Need Complicated Model Architectures For Temporal Networks?cited by 143
  10. 2023Temporal Graph Benchmark for Machine Learning on Temporal Graphscited by 138
  11. 2023Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learningcited by 133
  12. 2018dyngraph2vec: Capturing Network Dynamics using Dynamic Graph Representation Learningcited by 88
  13. 2020APAN: Asynchronous Propagation Attention Network for Real-time Temporal Graph Embeddingcited by 78
  14. 2021Continuous-Time Sequential Recommendation with Temporal Graph Collaborative Transformercited by 69
  15. 2023GraphAdapter: Tuning Vision-Language Models With Dual Knowledge Graphcited by 69
  16. 2021TCL: Transformer-based Dynamic Graph Modelling via Contrastive Learningcited by 68
  17. 2023Graph Neural Prompting with Large Language Modelscited by 64
  18. 2022TGL: A General Framework for Temporal GNN Training on Billion-Scale Graphscited by 61
  19. 2022Provably expressive temporal graph networkscited by 56
  20. 2022Neighborhood-aware Scalable Temporal Network Representation Learningcited by 54
  21. 2024Efficient Tuning and Inference for Large Language Models on Textual Graphscited by 54
  22. 2022TREND: TempoRal Event and Node Dynamics for Graph Representation Learningcited by 50
  23. 2023Patton: Language Model Pretraining on Text-Rich Networkscited by 49
  24. 2021Dynamic Network Embedding Surveycited by 44
  25. 2024Can Graph Learning Improve Planning in LLM-based Agents?cited by 43
  26. 2024Large Language Models-guided Dynamic Adaptation for Temporal Knowledge Graph Reasoningcited by 41
  27. 2023LLM4DyG: Can Large Language Models Solve Spatial-Temporal Problems on Dynamic Graphs?cited by 40
  28. 2024DTGB: A Comprehensive Benchmark for Dynamic Text-Attributed Graphscited by 27
  29. 2020Evolutionary dynamics of higher-order interactions in social networkscited by 26
  30. 2024Semi-supervised Credit Card Fraud Detection via Attribute-Driven Graph Representationcited by 26
  31. 2022Heterformer: Transformer-based Deep Node Representation Learning on Heterogeneous Text-Rich Networkscited by 24
  32. 2023Edgeformers: Graph-Empowered Transformers for Representation Learning on Textual-Edge Networkscited by 23
  33. 2024Improving Temporal Link Prediction via Temporal Walk Matrix Projectioncited by 21
  34. 2023TIGER: Temporal Interaction Graph Embedding with Restartscited by 17
  35. 2024GAugLLM: Improving Graph Contrastive Learning for Text-Attributed Graphs with Large Language Modelscited by 17
  36. 2024State Space Models on Temporal Graphs: A First-Principles Studycited by 16
  37. 2023RDGSL: Dynamic Graph Representation Learning with Structure Learningcited by 13
  38. 2024Node-Time Conditional Prompt Learning In Dynamic Graphscited by 11

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 3.

6 papers.

  1. 2026RAPO: Expanding Exploration for LLM Agents via Retrieval-Augmented Policy Optimizationcited by 6
  2. 2026Self-Evolving Agents as Dynamic Graph Transformation: A Survey and New Perspectivecited by 4
  3. 2025Global-Recent Semantic Reasoning on Dynamic Text-Attributed Graphs with Large Language Modelscited by 2
  4. 2025Rethinking Time Encoding via Learnable Transformation Functionscited by 1
  5. 2025GraphMatch: Fusing Language and Graph Representations in a Dynamic Two-Sided Work Marketplacecited by 0
  6. 2026Transfer Learning for Edge Classification on Dynamic Text-Attributed Graphscited by 0