mlsanityData to 14 Oct 2026

Relational Deep Learning: Challenges, Foundations and Next-Generation Architectures

arXiv 2506.16654 ↗First posted 19 Jun 2025cs.LG cs.AI cs.DBcited by 23

Not on the map. Why?

Abstract (arXiv)

Graph machine learning has led to a significant increase in the capabilities of models that learn on arbitrary graph-structured data and has been applied to molecules, social networks, recommendation systems, and transportation, among other domains. Data in multi-tabular relational databases can also be constructed as 'relational entity graphs' for Relational Deep Learning (RDL) - a new blueprint that enables end-to-end representation learning without traditional feature engineering. Compared to arbitrary graph-structured data, relational entity graphs have key properties: (i) their structure is defined by primary-foreign key relationships between entities in different tables, (ii) the structural connectivity is a function of the relational schema defining a database, and (iii) the graph connectivity is temporal and heterogeneous in nature. In this paper, we provide a comprehensive review of RDL by first introducing the representation of relational databases as relational entity graphs, and then reviewing public benchmark datasets that have been used to develop and evaluate recent GNN-based RDL models. We discuss key challenges including large-scale multi-table integration and the complexities of modeling temporal dynamics and heterogeneous data, while also surveying foundational neural network methods and recent architectural advances specialized for relational entity graphs. Finally, we explore opportunities to unify these distinct modeling challenges, highlighting how RDL converges multiple sub-fields in graph machine learning towards the design of foundation models that can transform the processing of relational data.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Builds on 58 of its 93 references that are in the map. First posted on arXiv 19 Jun 2025.

Builds on

58 of 93 references.

References · 58

  1. 2017Attention Is All You Needcited by 60,361
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  5. 2017Graph Attention Networkscited by 8,923
  6. 2017Inductive Representation Learning on Large Graphscited by 6,320
  7. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  8. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  9. 2017Neural Message Passing for Quantum Chemistrycited by 3,487
  10. 2023Qwen Technical Reportcited by 3,344
  11. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  12. 2016node2vec: Scalable Feature Learning for Networkscited by 1,918
38 more
  1. 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,694
  2. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,167
  3. 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,138
  4. 2018Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networkscited by 881
  5. 2019Heterogeneous Graph Attention Networkcited by 734
  6. 2022Recipe for a General, Powerful, Scalable Graph Transformercited by 713
  7. 2020Benchmarking Graph Neural Networkscited by 680
  8. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 674
  9. 2019TabNet: Attentive Interpretable Tabular Learningcited by 648
  10. 2020Heterogeneous Graph Transformercited by 582
  11. 2021Rethinking Graph Transformers with Spectral Attentioncited by 473
  12. 2020Inductive Representation Learning on Temporal Graphscited by 462
  13. 2020TabTransformer: Tabular Data Modeling Using Contextual Embeddingscited by 409
  14. 2018Invariant and Equivariant Graph Networkscited by 346
  15. 2021Graph Neural Networks with Learnable Structural and Positional Representationscited by 333
  16. 2019Time2Vec: Learning a Vector Representation of Timecited by 213
  17. 2021Inductive Representation Learning in Temporal Networks via Causal Anonymous Walkscited by 193
  18. 2022On Embeddings for Numerical Features in Tabular Deep Learningcited by 186
  19. 2023Towards Better Dynamic Graph Learning: New Architecture and Unified Librarycited by 157
  20. 2019AliGraph: A Comprehensive Graph Neural Network Platformcited by 153
  21. 2021GraphiT: Encoding Graph Structure in Transformerscited by 147
  22. 2023Temporal Graph Benchmark for Machine Learning on Temporal Graphscited by 138
  23. 2022NAGphormer: A Tokenized Graph Transformer for Node Classification in Large Graphscited by 137
  24. 2022ROLAND: Graph Learning Framework for Dynamic Graphscited by 132
  25. 2021ETA Prediction with Graph Neural Networks in Google Mapscited by 130
  26. 2019Stability Properties of Graph Neural Networkscited by 127
  27. 2020Graphon Neural Networks and the Transferability of Graph Neural Networkscited by 118
  28. 2024RelBench: A Benchmark for Deep Learning on Relational Databasescited by 96
  29. 2019Transferability of Spectral Graph Convolutional Neural Networkscited by 92
  30. 2023XTab: Cross-table Pretraining for Tabular Transformerscited by 81
  31. 2022Hierarchical Graph Transformer with Adaptive Node Samplingcited by 72
  32. 2024Reka Core, Flash, and Edge: A Series of Powerful Multimodal Language Modelscited by 67
  33. 2018On Weisfeiler-Leman Invariance: Subgraph Counts and Related Graph Propertiescited by 63
  34. 2021Gophormer: Ego-Graph Transformer for Node Classificationcited by 63
  35. 2022Scalable Spatiotemporal Graph Neural Networkscited by 52
  36. 2025RelGNN: Composite Message Passing for Relational Deep Learningcited by 48
  37. 2023Trompt: Towards a Better Deep Neural Network for Tabular Datacited by 36
  38. 2024On the Scalability of GNNs for Molecular Graphscited by 36

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 21.

23 papers.

Sort:

By area:

  1. 2025Relational Transformer: Toward Zero-Shot Foundation Models for Relational Datacited by 39
  2. 2026RelBench v2: A Large-Scale Benchmark and Repository for Relational Datacited by 20
  3. 2026PluRel: Synthetic Data unlocks Scaling Laws for Relational Foundation Modelscited by 19
  4. 2026No Need to Train Your RDB Foundation Modelcited by 10
  5. 2026Towards Anomaly Detection on Relational Datacited by 5
  6. 2026ReFuGe: Feature Generation for Prediction Tasks on Relational Databases with LLM Agentscited by 3
  7. 2025Database Views as Explanations for Relational Deep Learningcited by 1
  8. 2026RelPrism: A Multi-Faceted Pre-training Framework with Self-Generated Tasks for Relational Databasescited by 1
  9. 2026EmbedPart: Embedding-Driven Graph Partitioning for Scalable Graph Neural Network Trainingcited by 0
  10. 2026Gaussian Relational Graph Transformercited by 0
  11. 2026Towards Foundation Models for Relational Databases with Language Models and Graph Neural Networkscited by 0
  12. 2026Is Fixing Schema Graphs Necessary? Full-Resolution Graph Structure Learning for Relational Deep Learningcited by 0
11 more
  1. 2026RelGT-AC: A Relational Graph Transformer for Autocomplete Tasks in Relational Databasescited by 0
  2. 2026Structural Adversarial Attacks on Relational Deep Learning under Integrity Constraintscited by 0
  3. 2026UniSAGE: Unifying Static and Dynamic Attributes with Hyper-Structurecited by 0
  4. 2026RelShap: Relationally Consistent Shapley Explanationscited by 0
  5. 2026Incremental Evaluation and Training in Relational Deep Learningcited by 0
  6. 2026RAD: Rule-Augmented Relational Anomaly Detectioncited by 0
  7. 2026Relational Task Generation Language: A Declarative Specification Framework for Relational Deep Learningcited by 0
  8. 2026QUARTET: Quad-branch cross-Attention and Random-walk Traces for Enhancing Transformers on Relational Graphscited by 0
  9. 2026Temporal Heterogeneous Graph Pretraining for Relational Deep Learningcited by 0
  10. 2026The Universal Classifier for Graph Learningcited by 0
  11. 2026RT-J: Large-Scale Pretraining of Relational Transformers for Context-Efficient Predictionscited by 0