# Non-Homophilic Graph Pre-Training and Prompt Learning

arXiv: https://arxiv.org/abs/2408.12594 · HTML: https://mlsanity.com/paper/2408.12594/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 57 references that are in the map.
- First posted on arXiv 22 Aug 2024.

## Builds on (most cited first, 30 of 40)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Graph Neural Networks: A Review of Methods and Applications](https://mlsanity.com/paper/1812.08434/) (2018)
- [Variational Graph Auto-Encoders](https://mlsanity.com/paper/1611.07308/) (2016)
- [Making Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2012.15723/) (2020)
- [Graph Contrastive Learning with Augmentations](https://mlsanity.com/paper/2010.13902/) (2020)
- [Deep Graph Infomax](https://mlsanity.com/paper/1809.10341/) (2018)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [Strategies for Pre-training Graph Neural Networks](https://mlsanity.com/paper/1905.12265/) (2019)
- [Geom-GCN: Geometric Graph Convolutional Networks](https://mlsanity.com/paper/2002.05287/) (2020)
- [GPT Understands, Too](https://mlsanity.com/paper/2103.10385/) (2021)
- [Unified Language Model Pre-training for Natural Language Understanding and Generation](https://mlsanity.com/paper/1905.03197/) (2019)
- [It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners](https://mlsanity.com/paper/2009.07118/) (2020)
- [Contrastive Multi-View Representation Learning on Graphs](https://mlsanity.com/paper/2006.05582/) (2020)
- [Multi-scale Attributed Node Embedding](https://mlsanity.com/paper/1909.13021/) (2019)
- [Graph Transformer Networks](https://mlsanity.com/paper/1911.06455/) (2019)
- [GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training](https://mlsanity.com/paper/2006.09963/) (2020)
- [Beyond Low-frequency Information in Graph Convolutional Networks](https://mlsanity.com/paper/2101.00797/) (2021)
- [GPT-GNN: Generative Pre-Training of Graph Neural Networks](https://mlsanity.com/paper/2006.15437/) (2020)
- [Graph Representation Learning via Graphical Mutual Information Maximization](https://mlsanity.com/paper/2002.01169/) (2020)
- [Graph Neural Networks with Heterophily](https://mlsanity.com/paper/2009.13566/) (2020)
- [GraphPrompt: Unifying Pre-Training and Downstream Tasks for Graph Neural Networks](https://mlsanity.com/paper/2302.08043/) (2023)
- [Unified Vision and Language Prompt Learning](https://mlsanity.com/paper/2210.07225/) (2022)
- [Node Similarity Preserving Graph Convolutional Networks](https://mlsanity.com/paper/2011.09643/) (2020)

## Built on by (most cited first, 30 of 31)

- [SAMGPT: Text-free Graph Foundation Model for Multi-domain Pre-training and Cross-domain Adaptation](https://mlsanity.com/paper/2502.05424/) (2025)
- [RAGraph: A General Retrieval-Augmented Graph Learning Framework](https://mlsanity.com/paper/2410.23855/) (2024)
- [Edge Prompt Tuning for Graph Neural Networks](https://mlsanity.com/paper/2503.00750/) (2025)
- [GRAVER: Generative Graph Vocabularies for Robust Graph Foundation Models Fine-tuning](https://mlsanity.com/paper/2511.05592/) (2025)
- [DAGPrompT: Pushing the Limits of Graph Prompting with a Distribution-aware Graph Prompt Tuning Approach](https://mlsanity.com/paper/2501.15142/) (2025)
- [One Prompt Fits All: Universal Graph Adaptation for Pretrained Models](https://mlsanity.com/paper/2509.22416/) (2025)
- [Exploring the Potential of Large Language Models for Heterophilic Graphs](https://mlsanity.com/paper/2408.14134/) (2024)
- [GCoT: Chain-of-Thought Prompt Learning for Graphs](https://mlsanity.com/paper/2502.08092/) (2025)
- [Large Language Models Meet Text-Attributed Graphs: A Survey of Integration Frameworks and Applications](https://mlsanity.com/paper/2510.21131/) (2025)
- [HeTGB: A Comprehensive Benchmark for Heterophilic Text-Attributed Graphs](https://mlsanity.com/paper/2503.04822/) (2025)
- [GraphTOP: Graph Topology-Oriented Prompting for Graph Neural Networks](https://mlsanity.com/paper/2510.22451/) (2025)
- [Multi-Domain Riemannian Graph Gluing for Building Graph Foundation Models](https://mlsanity.com/paper/2603.00618/) (2026)
- [Graph Prompting for Graph Learning Models: Recent Advances and Future Directions](https://mlsanity.com/paper/2506.08326/) (2025)
- [Evaluating Progress in Graph Foundation Models: A Comprehensive Benchmark and New Insights](https://mlsanity.com/paper/2603.10033/) (2026)
- [Learning Graph Foundation Models on Riemannian Graph-of-Graphs](https://mlsanity.com/paper/2605.09993/) (2026)
- [Beyond Single-Granularity Prompts: A Multi-Scale Chain-of-Thought Prompt Learning for Graph](https://mlsanity.com/paper/2510.09394/) (2025)
- [Dynamic Graph Prompting via Topology-Routed Mixed-Curvature Experts](https://mlsanity.com/paper/2608.06031/) (2026)
- [When Prompting Meets Spiking: Graph Sparse Prompting via Spiking Graph Prompt Learning](https://mlsanity.com/paper/2601.02662/) (2026)
- [GFMate: Empowering Graph Foundation Models with Test-time Prompt Tuning](https://mlsanity.com/paper/2605.14809/) (2026)
- [Surprisingly Simple and Effective Multi-Domain Graph Foundation Model through Graph-to-Table Alignment](https://mlsanity.com/paper/2607.11374/) (2026)
- [Freeze, Prompt, and Adapt: A Framework for Source-free Unsupervised GNN Prompting](https://mlsanity.com/paper/2505.16903/) (2025)
- [Aligning the Spectrum: Hybrid Graph Pre-training and Prompt Tuning across Homophily and Heterophily](https://mlsanity.com/paper/2508.11328/) (2025)
- [MolGA: Molecular Graph Adaptation with Pre-trained 2D Graph Encoder](https://mlsanity.com/paper/2510.07289/) (2025)
- [Event-Aware Prompt Learning for Dynamic Graphs](https://mlsanity.com/paper/2510.11339/) (2025)
- [Adaptive Graph Mixture of Residual Experts: Unsupervised Learning on Diverse Graphs with Heterogeneous Specialization](https://mlsanity.com/paper/2510.21207/) (2025)
- [Robust Graph Fine-Tuning with Adversarial Graph Prompting](https://mlsanity.com/paper/2601.00229/) (2026)
- [GraphVec: Cross-Domain Graph Vectorization for Graph-Level Representation Learning](https://mlsanity.com/paper/2602.04244/) (2026)
- [A Unified Graph Language Model for Multi-Domain Multi-Task Graph Alignment Instruction Tuning](https://mlsanity.com/paper/2605.12197/) (2026)
- [Message Tuning Outshines Graph Prompt Tuning: A Prismatic Space Perspective](https://mlsanity.com/paper/2606.03290/) (2026)
- [Task-Specific Prompt with Global Context for Multi-Task Graph Pre-Training](https://mlsanity.com/paper/2609.00047/) (2026)

## Abstract (arXiv)

Graphs are ubiquitous for modeling complex relationships between objects across various fields. Graph neural networks (GNNs) have become a mainstream technique for graph-based applications, but their performance heavily relies on abundant labeled data. To reduce labeling requirement, pre-training and prompt learning has become a popular alternative. However, most existing prompt methods do not differentiate homophilic and heterophilic characteristics of real-world graphs. In particular, many real-world graphs are non-homophilic, not strictly or uniformly homophilic with mixing homophilic and heterophilic patterns, exhibiting varying non-homophilic characteristics across graphs and nodes. In this paper, we propose ProNoG, a novel pre-training and prompt learning framework for such non-homophilic graphs. First, we analyze existing graph pre-training methods, providing theoretical insights into the choice of pre-training tasks. Second, recognizing that each node exhibits unique non-homophilic characteristics, we propose a conditional network to characterize the node-specific patterns in downstream tasks. Finally, we thoroughly evaluate and analyze ProNoG through extensive experiments on ten public datasets.