# Relational Deep Learning: Challenges, Foundations and Next-Generation Architectures

arXiv: https://arxiv.org/abs/2506.16654 · HTML: https://mlsanity.com/paper/2506.16654/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 58 of its 93 references that are in the map.
- First posted on arXiv 19 Jun 2025.

## Builds on (most cited first, 30 of 58)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [Graph Convolutional Neural Networks for Web-Scale Recommender Systems](https://mlsanity.com/paper/1806.01973/) (2018)
- [Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks](https://mlsanity.com/paper/1810.02244/) (2018)
- [Heterogeneous Graph Attention Network](https://mlsanity.com/paper/1903.07293/) (2019)
- [Recipe for a General, Powerful, Scalable Graph Transformer](https://mlsanity.com/paper/2205.12454/) (2022)
- [Benchmarking Graph Neural Networks](https://mlsanity.com/paper/2003.00982/) (2020)
- [TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second](https://mlsanity.com/paper/2207.01848/) (2022)
- [TabNet: Attentive Interpretable Tabular Learning](https://mlsanity.com/paper/1908.07442/) (2019)
- [Heterogeneous Graph Transformer](https://mlsanity.com/paper/2003.01332/) (2020)
- [Rethinking Graph Transformers with Spectral Attention](https://mlsanity.com/paper/2106.03893/) (2021)
- [Inductive Representation Learning on Temporal Graphs](https://mlsanity.com/paper/2002.07962/) (2020)
- [TabTransformer: Tabular Data Modeling Using Contextual Embeddings](https://mlsanity.com/paper/2012.06678/) (2020)
- [Invariant and Equivariant Graph Networks](https://mlsanity.com/paper/1812.09902/) (2018)
- [Graph Neural Networks with Learnable Structural and Positional Representations](https://mlsanity.com/paper/2110.07875/) (2021)
- [Time2Vec: Learning a Vector Representation of Time](https://mlsanity.com/paper/1907.05321/) (2019)
- [Inductive Representation Learning in Temporal Networks via Causal Anonymous Walks](https://mlsanity.com/paper/2101.05974/) (2021)
- [On Embeddings for Numerical Features in Tabular Deep Learning](https://mlsanity.com/paper/2203.05556/) (2022)

## Built on by (most cited first, 23 of 23)

- [Relational Transformer: Toward Zero-Shot Foundation Models for Relational Data](https://mlsanity.com/paper/2510.06377/) (2025)
- [RelBench v2: A Large-Scale Benchmark and Repository for Relational Data](https://mlsanity.com/paper/2602.12606/) (2026)
- [PluRel: Synthetic Data unlocks Scaling Laws for Relational Foundation Models](https://mlsanity.com/paper/2602.04029/) (2026)
- [No Need to Train Your RDB Foundation Model](https://mlsanity.com/paper/2602.13697/) (2026)
- [Towards Anomaly Detection on Relational Data](https://mlsanity.com/paper/2606.18621/) (2026)
- [ReFuGe: Feature Generation for Prediction Tasks on Relational Databases with LLM Agents](https://mlsanity.com/paper/2601.17735/) (2026)
- [Database Views as Explanations for Relational Deep Learning](https://mlsanity.com/paper/2509.09482/) (2025)
- [RelPrism: A Multi-Faceted Pre-training Framework with Self-Generated Tasks for Relational Databases](https://mlsanity.com/paper/2605.23241/) (2026)
- [EmbedPart: Embedding-Driven Graph Partitioning for Scalable Graph Neural Network Training](https://mlsanity.com/paper/2604.01000/) (2026)
- [Gaussian Relational Graph Transformer](https://mlsanity.com/paper/2605.15575/) (2026)
- [Towards Foundation Models for Relational Databases with Language Models and Graph Neural Networks](https://mlsanity.com/paper/2605.16085/) (2026)
- [Is Fixing Schema Graphs Necessary? Full-Resolution Graph Structure Learning for Relational Deep Learning](https://mlsanity.com/paper/2605.21475/) (2026)
- [RelGT-AC: A Relational Graph Transformer for Autocomplete Tasks in Relational Databases](https://mlsanity.com/paper/2606.03040/) (2026)
- [Structural Adversarial Attacks on Relational Deep Learning under Integrity Constraints](https://mlsanity.com/paper/2607.07089/) (2026)
- [UniSAGE: Unifying Static and Dynamic Attributes with Hyper-Structure](https://mlsanity.com/paper/2607.14102/) (2026)
- [RelShap: Relationally Consistent Shapley Explanations](https://mlsanity.com/paper/2608.11508/) (2026)
- [Incremental Evaluation and Training in Relational Deep Learning](https://mlsanity.com/paper/2608.13023/) (2026)
- [RAD: Rule-Augmented Relational Anomaly Detection](https://mlsanity.com/paper/2608.23468/) (2026)
- [Relational Task Generation Language: A Declarative Specification Framework for Relational Deep Learning](https://mlsanity.com/paper/2609.01292/) (2026)
- [QUARTET: Quad-branch cross-Attention and Random-walk Traces for Enhancing Transformers on Relational Graphs](https://mlsanity.com/paper/2609.26855/) (2026)
- [Temporal Heterogeneous Graph Pretraining for Relational Deep Learning](https://mlsanity.com/paper/2609.35219/) (2026)
- [The Universal Classifier for Graph Learning](https://mlsanity.com/paper/2609.36302/) (2026)
- [RT-J: Large-Scale Pretraining of Relational Transformers for Context-Efficient Predictions](https://mlsanity.com/paper/2610.17867/) (2026)

## Abstract (arXiv)

Graph machine learning has led to a significant increase in the capabilities of models that learn on arbitrary graph-structured data and has been applied to molecules, social networks, recommendation systems, and transportation, among other domains. Data in multi-tabular relational databases can also be constructed as 'relational entity graphs' for Relational Deep Learning (RDL) - a new blueprint that enables end-to-end representation learning without traditional feature engineering. Compared to arbitrary graph-structured data, relational entity graphs have key properties: (i) their structure is defined by primary-foreign key relationships between entities in different tables, (ii) the structural connectivity is a function of the relational schema defining a database, and (iii) the graph connectivity is temporal and heterogeneous in nature. In this paper, we provide a comprehensive review of RDL by first introducing the representation of relational databases as relational entity graphs, and then reviewing public benchmark datasets that have been used to develop and evaluate recent GNN-based RDL models. We discuss key challenges including large-scale multi-table integration and the complexities of modeling temporal dynamics and heterogeneous data, while also surveying foundational neural network methods and recent architectural advances specialized for relational entity graphs. Finally, we explore opportunities to unify these distinct modeling challenges, highlighting how RDL converges multiple sub-fields in graph machine learning towards the design of foundation models that can transform the processing of relational data.