mlsanityData to 14 Oct 2026

Transformers Meet Relational Databases

arXiv 2412.05218 ↗First posted 6 Dec 2024cs.LG cs.DBcited by 9

Not on the map. Why?

Abstract (arXiv)

Transformer models have continuously expanded into all machine learning domains convertible to the underlying sequence-to-sequence representation, including tabular data. However, while ubiquitous, this representation restricts their extension to the more general case of relational databases. In this paper, we introduce a modular neural message-passing scheme that closely adheres to the formal relational model, enabling direct end-to-end learning of tabular Transformers from database storage systems. We address the challenges of appropriate learning data representation and loading, which are critical in the database setting, and compare our approach against a number of representative models from various related fields across a significantly wide range of datasets. Our results demonstrate a superior performance of this newly proposed class of neural architectures.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 32 of its 73 references that are in the map. First posted on arXiv 6 Dec 2024.

Builds on

32 of 73 references.

References · 32

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  5. 2017Inductive Representation Learning on Large Graphscited by 6,320
  6. 2016Layer Normalizationcited by 5,669
  7. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  8. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,919
  9. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
  10. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  11. 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,694
  12. 2022Recipe for a General, Powerful, Scalable Graph Transformercited by 713
20 more
  1. 2019TabNet: Attentive Interpretable Tabular Learningcited by 648
  2. 2020Heterogeneous Graph Transformercited by 582
  3. 2021Tabular Data: Deep Learning is Not All You Needcited by 535
  4. 2017Ray: A Distributed Framework for Emerging AI Applicationscited by 527
  5. 2019Graph Transformer Networkscited by 523
  6. 2018Hypergraph Neural Networkscited by 488
  7. 2020TabTransformer: Tabular Data Modeling Using Contextual Embeddingscited by 409
  8. 2021SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Trainingcited by 319
  9. 2021Well-tuned Simple Nets Excel on Tabular Datasetscited by 151
  10. 2023Tracr: Compiled Transformers as a Laboratory for Interpretabilitycited by 104
  11. 2024RelBench: A Benchmark for Deep Learning on Relational Databasescited by 96
  12. 2020Supervised Learning on Relational Databases with Graph Neural Networkscited by 48
  13. 2020Turning 30: New Ideas in Inductive Logic Programmingcited by 42
  14. 2023Trompt: Towards a Better Deep Neural Network for Tabular Datacited by 36
  15. 2023ExcelFormer: A neural network surpassing GBDTs on tabular datacited by 34
  16. 2024PyTorch Frame: A Modular Framework for Multi-Modal Tabular Learningcited by 32
  17. 2023GFS: Graph-based Feature Synthesis for Prediction over Relational Databasescited by 16
  18. 2023Towards Foundation Models for Relational Databases [Vision Paper]cited by 7
  19. 2018Explaining Queries over Web Tables to Non-Expertscited by 5
  20. 2023SPARE: A Single-Pass Neural Model for Relational Databasescited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 9.

9 papers.

  1. 2026RelBench v2: A Large-Scale Benchmark and Repository for Relational Datacited by 20
  2. 2026No Need to Train Your RDB Foundation Modelcited by 10
  3. 2026End-to-End Compression for Tabular Foundation Modelscited by 6
  4. 2026A Fair Benchmarking of Deep Relational Database Learning Modelscited by 1
  5. 2026Universal Encoders for Modular Relational Deep Learningcited by 0
  6. 2026Incremental Evaluation and Training in Relational Deep Learningcited by 0
  7. 2026Relational Task Generation Language: A Declarative Specification Framework for Relational Deep Learningcited by 0
  8. 2026STEER: Reducing Inference Cost in Relational Foundation Models through Semantically Informed Samplingcited by 0
  9. 2026RelICL: Training-free Relational Learning with Tabular Foundation Modelscited by 0