# Towards Graph Foundation Models for Personalization

arXiv: https://arxiv.org/abs/2403.07478 · HTML: https://mlsanity.com/paper/2403.07478/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 25 references that are in the map.
- First posted on arXiv 12 Mar 2024.

## Builds on (most cited first, 18 of 18)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Graph Convolutional Neural Networks for Web-Scale Recommender Systems](https://mlsanity.com/paper/1806.01973/) (2018)
- [Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks](https://mlsanity.com/paper/2311.06242/) (2023)
- [Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)](https://mlsanity.com/paper/2203.13366/) (2022)
- [Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Action](https://mlsanity.com/paper/2312.17172/) (2023)
- [GreaseLM: Graph REASoning Enhanced Language Models for Question Answering](https://mlsanity.com/paper/2201.08860/) (2022)
- [Graph Foundation Models: Concepts, Opportunities and Challenges](https://mlsanity.com/paper/2310.11829/) (2023)
- [CoLLM: Integrating Collaborative Embeddings into Large Language Models for Recommendation](https://mlsanity.com/paper/2310.19488/) (2023)
- [Towards Foundation Models for Knowledge Graph Reasoning](https://mlsanity.com/paper/2310.04562/) (2023)
- [LLM-Rec: Personalized Recommendation via Prompting Large Language Models](https://mlsanity.com/paper/2307.15780/) (2023)
- [Relational Deep Learning: Graph Representation Learning on Relational Databases](https://mlsanity.com/paper/2312.04615/) (2023)
- [Graph-Aware Language Model Pre-Training on a Large Graph Corpus Can Help Multiple Graph Applications](https://mlsanity.com/paper/2306.02592/) (2023)
- [Grale: Designing Networks for Graph Learning](https://mlsanity.com/paper/2007.12002/) (2020)
- [TwHIN: Embedding the Twitter Heterogeneous Information Network for Personalized Recommendation](https://mlsanity.com/paper/2202.05387/) (2022)
- [Personalized Audiobook Recommendations at Spotify Through Graph Neural Networks](https://mlsanity.com/paper/2403.05185/) (2024)
- [Efficient and effective training of language and graph neural network models](https://mlsanity.com/paper/2206.10781/) (2022)

## Built on by (most cited first, 12 of 12)

- [A Survey of Foundation Model-Powered Recommender Systems: From Feature-Based, Generative to Agentic Paradigms](https://mlsanity.com/paper/2504.16420/) (2025)
- [Question-guided Knowledge Graph Re-scoring and Injection for Knowledge Graph Question Answering](https://mlsanity.com/paper/2410.01401/) (2024)
- [An Automatic Graph Construction Framework based on Large Language Models for Recommendation](https://mlsanity.com/paper/2412.18241/) (2024)
- [Enhancing High-order Interaction Awareness in LLM-based Recommender Model](https://mlsanity.com/paper/2409.19979/) (2024)
- [Are We Measuring Oversmoothing in Graph Neural Networks Correctly?](https://mlsanity.com/paper/2502.04591/) (2025)
- [IOHunter: Graph Foundation Model to Uncover Online Information Operations](https://mlsanity.com/paper/2412.14663/) (2024)
- [Graph Foundation Models for Recommendation: A Comprehensive Survey](https://mlsanity.com/paper/2502.08346/) (2025)
- [Stuart-Landau Oscillatory Graph Neural Network](https://mlsanity.com/paper/2511.08094/) (2025)
- [A Scalable and Efficient Signal Integration System for Job Matching](https://mlsanity.com/paper/2507.09797/) (2025)
- [Transfer Learning for Temporal Link Prediction](https://mlsanity.com/paper/2504.10925/) (2025)
- [RankGraph: Unified Heterogeneous Graph Learning for Cross-Domain Recommendation](https://mlsanity.com/paper/2509.02942/) (2025)
- [NeuroPathNet: Dynamic Path Trajectory Learning for Brain Functional Connectivity Analysis](https://mlsanity.com/paper/2510.24025/) (2025)

## Abstract (arXiv)

In the realm of personalization, integrating diverse information sources such as consumption signals and content-based representations is becoming increasingly critical to build state-of-the-art solutions. In this regard, two of the biggest trends in research around this subject are Graph Neural Networks (GNNs) and Foundation Models (FMs). While GNNs emerged as a popular solution in industry for powering personalization at scale, FMs have only recently caught attention for their promising performance in personalization tasks like ranking and retrieval. In this paper, we present a graph-based foundation modeling approach tailored to personalization. Central to this approach is a Heterogeneous GNN (HGNN) designed to capture multi-hop content and consumption relationships across a range of recommendable item types. To ensure the generality required from a Foundation Model, we employ a Large Language Model (LLM) text-based featurization of nodes that accommodates all item types, and construct the graph using co-interaction signals, which inherently transcend content specificity. To facilitate practical generalization, we further couple the HGNN with an adaptation mechanism based on a two-tower (2T) architecture, which also operates agnostically to content type. This multi-stage approach ensures high scalability; while the HGNN produces general purpose embeddings, the 2T component models in a continuous space the sheer size of user-item interaction data. Our comprehensive approach has been rigorously tested and proven effective in delivering recommendations across a diverse array of products within a real-world, industrial audio streaming platform.