mlsanityData to 7 Oct 2026

Can Graph Neural Networks Learn Language with Extremely Weak Text Supervision?

arXiv 2412.08174 ↗First posted 11 Dec 2024cs.LG cs.AI cs.SIcited by 13

Not on the map. Why?

Abstract (arXiv)

While great success has been achieved in building vision models with Contrastive Language-Image Pre-training (CLIP) over internet-scale image-text pairs, building transferable Graph Neural Networks (GNNs) with CLIP pipeline is challenging because of the scarcity of labeled data and text supervision, different levels of downstream tasks, and the conceptual gaps between domains. In this work, to address these issues, we propose a multi-modal prompt learning paradigm to effectively adapt pre-trained GNN to downstream tasks and data, given only a few semantically labeled samples, each with extremely weak text supervision. Our new paradigm embeds the graphs directly in the same space as the Large Language Models (LLMs) by learning both graph prompts and text prompts simultaneously. We demonstrate the superior performance of our paradigm in few-shot, multi-task-level, and cross-domain settings. Moreover, we build the first CLIP-style zero-shot classification prototype that can generalize GNNs to unseen classes with extremely weak text supervision. The code is available at https://github.com/Violet24K/Morpher.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 116 of its 169 references that are in the map. First posted on arXiv 11 Dec 2024.

Builds on

116 of 169 references.

References · 116

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  5. 2020Scaling Laws for Neural Language Modelscited by 6,411
  6. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  7. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  8. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  9. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  10. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  11. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  12. 2020Supervised Contrastive Learningcited by 2,803
38 more
  1. 2023A Survey of Large Language Modelscited by 2,537
  2. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
  3. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  4. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,427
  5. 2020Graph Contrastive Learning with Augmentationscited by 1,203
  6. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,037
  7. 2016Revisiting Semi-Supervised Learning with Graph Embeddingscited by 970
  8. 2019Strategies for Pre-training Graph Neural Networkscited by 964
  9. 2022Galactica: A Large Language Model for Sciencecited by 736
  10. 2020TUDataset: A collection of benchmark datasets for learning with graphscited by 718
  11. 2022MaPLe: Multi-modal Prompt Learningcited by 685
  12. 2020Benchmarking Graph Neural Networkscited by 672
  13. 2020Contrastive Multi-View Representation Learning on Graphscited by 651
  14. 2021VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understandingcited by 596
  15. 2019Graph Transformer Networkscited by 520
  16. 2022DiGress: Discrete Denoising diffusion for graph generationcited by 503
  17. 2021Rethinking Graph Transformers with Spectral Attentioncited by 468
  18. 2020GCC: Graph Contrastive Coding for Graph Neural Network Pre-Trainingcited by 461
  19. 2021PointCLIP: Point Cloud Understanding by CLIPcited by 461
  20. 2022GraphMAE: Self-Supervised Masked Graph Autoencoderscited by 443
  21. 2019InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximizationcited by 441
  22. 2021AudioCLIP: Extending CLIP to Image, Text and Audiocited by 335
  23. 2020Improving Graph Neural Network Expressivity via Subgraph Isomorphism Countingcited by 312
  24. 2020GPT-GNN: Generative Pre-Training of Graph Neural Networkscited by 302
  25. 2021Graph Self-Supervised Learning: A Surveycited by 289
  26. 2021Wav2CLIP: Learning Robust Audio Representations From CLIPcited by 210
  27. 2023Large Language Models on Graphs: A Comprehensive Surveycited by 190
  28. 2023HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolutioncited by 187
  29. 2022SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentationcited by 171
  30. 2022GreaseLM: Graph REASoning Enhanced Language Models for Question Answeringcited by 170
  31. 2023GraphPrompt: Unifying Pre-Training and Downstream Tasks for Graph Neural Networkscited by 166
  32. 2022Learning on Large-scale Text-attributed Graphs via Variational Inferencecited by 153
  33. 2023All in One: Multi-task Prompting for Graph Neural Networkscited by 147
  34. 2021GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graphcited by 143
  35. 2021Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Predictioncited by 137
  36. 2022A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Languagecited by 129
  37. 2023MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adaptercited by 128
  38. 2020ProGen: Language Modeling for Protein Generationcited by 127

50 most cited of 116 All 116 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 10; 2026: 1.

13 papers.

Sort:

  1. 2025Language in the Flow of Time: Time-Series-Paired Texts Weaved into a Unified Temporal Narrativecited by 35
  2. 2025SelfElicit: Your Language Model Secretly Knows Where is the Relevant Evidencecited by 18
  3. 2025Graph4MM: Weaving Multimodal Learning with Structural Informationcited by 17
  4. 2024PyG-SSL: A Graph Self-Supervised Learning Toolkitcited by 14
  5. 2024APEX2: Adaptive and Extreme Summarization for Personalized Knowledge Graphscited by 13
  6. 2025Adaptive Fault Tolerance Mechanisms of Large Language Models in Cloud Computing Environmentscited by 12
  7. 2025Research on Large Language Model Cross-Cloud Privacy Protection and Collaborative Training based on Federated Learningcited by 11
  8. 2025Harnessing Consistency for Robust Test-Time LLM Ensemblecited by 10
  9. 2025Data Augmentation Through Random Style Replacementcited by 9
  10. 2025ClimateBench-M: A Multi-Modal Climate Data Benchmark with a Simple Generative Methodcited by 4
  11. 2025Learnable Spatial-Temporal Positional Encoding for Link Predictioncited by 3
  12. 2025HeroFilter: Adaptive Spectral Graph Filter for Varying Heterophilic Relationscited by 3
1 more
  1. 2026OWLEYE: Zero-Shot Learner for Cross-Domain Graph Data Anomaly Detectioncited by 1