mlsanityData to 14 Oct 2026

TAGA: Text-Attributed Graph Self-Supervised Learning by Synergizing Graph and Text Mutual Transformations

arXiv 2405.16800 ↗First posted 27 May 2024cs.LG cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Text-Attributed Graphs (TAGs) enhance graph structures with natural language descriptions, enabling detailed representation of data and their relationships across a broad spectrum of real-world scenarios. Despite the potential for deeper insights, existing TAG representation learning primarily relies on supervised methods, necessitating extensive labeled data and limiting applicability across diverse contexts. This paper introduces a new self-supervised learning framework, Text-And-Graph Multi-View Alignment (TAGA), which overcomes these constraints by integrating TAGs' structural and semantic dimensions. TAGA constructs two complementary views: Text-of-Graph view, which organizes node texts into structured documents based on graph topology, and the Graph-of-Text view, which converts textual nodes and connections into graph data. By aligning representations from both views, TAGA captures joint textual and structural information. In addition, a novel structure-preserving random walk algorithm is proposed for efficient training on large-sized TAGs. Our framework demonstrates strong performance in zero-shot and few-shot scenarios across eight real-world datasets.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 37 of its 47 references that are in the map. First posted on arXiv 27 May 2024.

Builds on

37 of 47 references.

References · 37

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  4. 2017Inductive Representation Learning on Large Graphscited by 6,320
  5. 2018How Powerful are Graph Neural Networks?cited by 4,393
  6. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  7. 2016Variational Graph Auto-Encoderscited by 1,382
  8. 2020Graph Contrastive Learning with Augmentationscited by 1,208
  9. 2018Deep Graph Infomaxcited by 1,106
  10. 2018Pitfalls of Graph Neural Network Evaluationcited by 1,007
  11. 2016Revisiting Semi-Supervised Learning with Graph Embeddingscited by 976
  12. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 740
25 more
  1. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 601
  2. 2018FastGCN: Fast Learning with Graph Convolutional Networks via Importance Samplingcited by 584
  3. 2020Deep Graph Contrastive Representation Learningcited by 539
  4. 2020Graph Neural Networks in Recommender Systems: A Surveycited by 494
  5. 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compressioncited by 453
  6. 2022GraphMAE: Self-Supervised Masked Graph Autoencoderscited by 448
  7. 2023Exploring the Potential of Large Language Models (LLMs) in Learning on Graphscited by 274
  8. 2023Can Language Models Solve Graph Problems in Natural Language?cited by 252
  9. 2023LongLoRA: Efficient Fine-tuning of Long-Context Large Language Modelscited by 244
  10. 2023GraphGPT: Graph Instruction Tuning for Large Language Modelscited by 226
  11. 2023Language is All a Graph Needscited by 220
  12. 2023LongNet: Scaling Transformers to 1,000,000,000 Tokenscited by 204
  13. 2023Large Language Models on Graphs: A Comprehensive Surveycited by 192
  14. 2023Talk like a Graph: Encoding Graphs for Large Language Modelscited by 176
  15. 2022Learning on Large-scale Text-attributed Graphs via Variational Inferencecited by 155
  16. 2021GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graphcited by 143
  17. 2023Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learningcited by 133
  18. 2023AnglE-optimized Text Embeddingscited by 99
  19. 2023Augmenting Low-Resource Text Classification with Graph-Grounded Pre-training and Promptingcited by 62
  20. 2021A Survey of Machine Learning for Computer Architecture and Systemscited by 53
  21. 2023Patton: Language Model Pretraining on Text-Rich Networkscited by 49
  22. 2023GRENADE: Graph-Centric Language Model for Self-Supervised Representation Learning on Text-Attributed Graphscited by 28
  23. 2023Beyond Text: A Deep Dive into Large Language Models' Ability on Understanding Graph Datacited by 25
  24. 2023Edgeformers: Graph-Empowered Transformers for Representation Learning on Textual-Edge Networkscited by 23
  25. 2022Eliciting Structural and Semantic Global Knowledge in Unsupervised Graph Contrastive Learningcited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 2; 2026: 1.

5 papers.

  1. 2025LLM as GNN: Graph Vocabulary Learning for Text-Attributed Graph Foundation Modelscited by 22
  2. 2024Graph Learning in the Era of LLMs: A Survey from the Perspective of Data, Models, and Taskscited by 10
  3. 2024MAO: A Framework for Process Model Generation with Multi-Agent Orchestrationcited by 8
  4. 2025Dynamic Bundling with Large Language Models for Zero-Shot Inference on Text-Attributed Graphscited by 3
  5. 2026TERGAD: Structure-Aware Text-Enhanced Representations for Graph Anomaly Detectioncited by 0