# Training Graph Foundation Models on The Web Graph

arXiv: https://arxiv.org/abs/2609.30894 · HTML: https://mlsanity.com/paper/2609.30894/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 37 of its 44 references that are in the map.
- First posted on arXiv 25 Sep 2026.

## Builds on (most cited first, 30 of 37)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model](https://mlsanity.com/paper/2502.02737/) (2025)
- [One for All: Towards Training One Graph Model for All Classification Tasks](https://mlsanity.com/paper/2310.00149/) (2023)
- [Random Features Strengthen Graph Neural Networks](https://mlsanity.com/paper/2002.03155/) (2020)
- [A Survey on The Expressive Power of Graph Neural Networks](https://mlsanity.com/paper/2003.04078/) (2020)
- [Graph Foundation Models: Concepts, Opportunities and Challenges](https://mlsanity.com/paper/2310.11829/) (2023)
- [PRODIGY: Enabling In-context Learning Over Graphs](https://mlsanity.com/paper/2305.12600/) (2023)
- [Position: Graph Foundation Models are Already Here](https://mlsanity.com/paper/2402.02216/) (2024)
- [OpenGraph: Towards Open Graph Foundation Models](https://mlsanity.com/paper/2403.01121/) (2024)
- [Approximation Ratios of Graph Neural Networks for Combinatorial Problems](https://mlsanity.com/paper/1905.10261/) (2019)
- [UniGraph: Learning a Unified Cross-Domain Foundation Model for Text-Attributed Graphs](https://mlsanity.com/paper/2402.13630/) (2024)
- [SAMGPT: Text-free Graph Foundation Model for Multi-domain Pre-training and Cross-domain Adaptation](https://mlsanity.com/paper/2502.05424/) (2025)
- [GOFA: A Generative One-For-All Model for Joint Graph Language Modeling](https://mlsanity.com/paper/2407.09709/) (2024)
- [AnyGraph: Graph Foundation Model in the Wild](https://mlsanity.com/paper/2408.10700/) (2024)
- [UniGraph2: Learning a Unified Embedding Space to Bind Multimodal Graphs](https://mlsanity.com/paper/2502.00806/) (2025)
- [Multi-Domain Graph Foundation Models: Robust Knowledge Transfer via Topology Alignment](https://mlsanity.com/paper/2502.02017/) (2025)
- [Turning Tabular Foundation Models into Graph Foundation Models](https://mlsanity.com/paper/2508.20906/) (2025)
- [Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Trees](https://mlsanity.com/paper/2412.16441/) (2024)
- [GraphICL: Unlocking Graph Learning Potential in LLMs through Structured Prompt Design](https://mlsanity.com/paper/2501.15755/) (2025)
- [GraphPFN: A Prior-Data Fitted Graph Foundation Model](https://mlsanity.com/paper/2509.21489/) (2025)
- [Graph Generative Pre-trained Transformer](https://mlsanity.com/paper/2501.01073/) (2025)
- [GraphFM: A generalist graph transformer that learns transferable representations across diverse domains](https://mlsanity.com/paper/2407.11907/) (2024)
- [Flatten Graphs as Sequences: Transformers are Scalable Graph Generators](https://mlsanity.com/paper/2502.02216/) (2025)
- [Are Large Language Models In-Context Graph Learners?](https://mlsanity.com/paper/2502.13562/) (2025)
- [Can TabPFN Compete with GNNs for Node Classification via Graph Tabularization?](https://mlsanity.com/paper/2512.08798/) (2025)
- [Learning Posterior Predictive Distributions for Node Classification from Synthetic Graph Priors](https://mlsanity.com/paper/2604.19028/) (2026)
- [Towards A Universal Graph Structural Encoder](https://mlsanity.com/paper/2504.10917/) (2025)
- [TFMLinker: Universal Link Predictor by Graph In-Context Learning with Tabular Foundation Models](https://mlsanity.com/paper/2602.08592/) (2026)
- [Billion-Scale Graph Foundation Models](https://mlsanity.com/paper/2602.04768/) (2026)

## Abstract (arXiv)

We introduce Acacia, a graph foundation model, trained on the web graph. Acacia (i) supports arbitrary feature dimensionalities and semantics without additional training, (ii) supports a wide range of tasks, including node classification, link prediction, node clustering, and graph generation, without additional training, (iii) has in-context learning capabilities, and (iv) does not rely on pretrained LLMs. In particular, existing graph foundation models often require training additional classification heads or feature projectors to accommodate new graphs or new labels, whereas Acacia does not. Moreover, existing graph foundation models often gain their capabilities by being stitched together with pretrained LLMs, whereas Acacia is trained from scratch using only the Common Crawl web graph. This is also an important result because it provides evidence that graph models can acquire emergent capabilities from scratch like LLMs.