mlsanityData to 14 Oct 2026

Towards Foundation Models for Relational Databases [Vision Paper]

arXiv 2305.15321 ↗First posted 24 May 2023cs.DB cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

Tabular representation learning has recently gained a lot of attention. However, existing approaches only learn a representation from a single table, and thus ignore the potential to learn from the full structure of relational databases, including neighboring tables that can contain important information for a contextualized representation. Moreover, current models are significantly limited in scale, which prevents that they learn from large databases. In this paper, we thus introduce our vision of relational representation learning, that can not only learn from the full relational structure, but also can scale to larger database sizes that are commonly found in real-world. Moreover, we also discuss opportunities and challenges we see along the way to enable this vision and present initial very promising results. Overall, we argue that this direction can lead to foundation models for relational databases that are today only available for text and images.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 14 of its 26 references that are in the map. First posted on arXiv 24 May 2023.

Builds on

14 of 26 references.

References · 14

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  3. 2017Inductive Representation Learning on Large Graphscited by 6,320
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  5. 2020TaBERT: Pretraining for Joint Understanding of Textual and Tabular Datacited by 398
  6. 2019KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representationcited by 356
  7. 2022Can Foundation Models Wrangle Your Data?cited by 175
  8. 2021Directed Acyclic Graph Neural Networkscited by 70
  9. 2020Supervised Learning on Relational Databases with Graph Neural Networkscited by 48
  10. 2021GitTables: A Large-Scale Corpus of Relational Tablescited by 45
  11. 2019Deep Unsupervised Cardinality Estimationcited by 36
  12. 2020RPT: Relational Pre-trained Transformer Is Almost All You Need towards Democratizing Data Preparationcited by 27
2 more
  1. 2022Efficient and effective training of language and graph neural network modelscited by 9
  2. 2019Termite: A System for Tunneling Through Heterogeneous Datacited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 1; 2026: 3.

7 papers.

  1. 2023Large Language Models and Knowledge Graphs: Opportunities and Challengescited by 71
  2. 2025Relational Transformer: Toward Zero-Shot Foundation Models for Relational Datacited by 39
  3. 2024Graph Neural Networks for Tabular Data Learning: A Survey with Taxonomy and Directionscited by 13
  4. 2024Transformers Meet Relational Databasescited by 9
  5. 2026Towards Foundation Models for Relational Databases with Language Models and Graph Neural Networkscited by 0
  6. 2026LLMs+Graphs: Toward Graph-Native, Synergistic AI Systemscited by 0
  7. 2026RT-J: Large-Scale Pretraining of Relational Transformers for Context-Efficient Predictionscited by 0