mlsanityData to 7 Oct 2026

Relational In-Context Learning via Synthetic Pre-training with Structural Prior

arXiv 2603.03805 ↗First posted 4 Mar 2026cs.LG cs.AI cs.DBcited by 11

Not on the map. Why?

Abstract (arXiv)

Relational Databases (RDBs) are the backbone of modern business, yet they lack foundation models comparable to those in text or vision. A key obstacle is that high-quality RDBs are private, scarce, and structurally heterogeneous, making internet-scale pre-training infeasible. To overcome this data scarcity, we introduce RDB-PFN, the first relational foundation model trained purely via synthetic data. Inspired by Prior-Data Fitted Networks (PFNs), where synthetic data generated from Structural Causal Models (SCMs) enables reasoning on single tables, we design a Relational Prior Generator to create an infinite stream of diverse RDBs from scratch. Pre-training on over 2 million synthetic single-table and relational tasks, RDB-PFN learns to adapt to any new database instantly via genuine in-context learning. Experiments show that RDB-PFN achieves strong few-shot performance on 19 real-world relational prediction tasks, outperforming state-of-the-art tabular foundation models evaluated on the same DFS-linearized inputs, while using a lightweight architecture and fast inference. The code is available at https://github.com/MuLabPKU/RDBPFN.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 54 of its 77 references that are in the map. First posted on arXiv 4 Mar 2026.

Builds on

54 of 77 references.

References · 54

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2020Scaling Laws for Neural Language Modelscited by 6,411
  5. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  6. 2016node2vec: Scalable Feature Learning for Networkscited by 1,913
  7. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  8. 2021Revisiting Deep Learning Models for Tabular Datacited by 725
  9. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
  10. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 643
  11. 2019TabNet: Attentive Interpretable Tabular Learningcited by 640
  12. 2020TabTransformer: Tabular Data Modeling Using Contextual Embeddingscited by 398
38 more
  1. 2020AutoGluon-Tabular: Robust and Accurate AutoML for Structured Datacited by 368
  2. 2021Transformers Can Do Bayesian Inferencecited by 338
  3. 2021SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Trainingcited by 310
  4. 2025TabICL: A Tabular Foundation Model for In-Context Learning on Large Datacited by 267
  5. 2022On Embeddings for Numerical Features in Tabular Deep Learningcited by 185
  6. 2024The Road Less Scheduledcited by 158
  7. 2020Handling Missing Data with Graph Representation Learningcited by 132
  8. 2022TransTab: Learning Transferable Tabular Transformers Across Tablescited by 132
  9. 2025TabPFN-2.5: Advancing the State of the Art in Tabular Foundation Modelscited by 115
  10. 2024TabM: Advancing Tabular Deep Learning with Parameter-Efficient Ensemblingcited by 105
  11. 2023ForecastPFN: Synthetically-Trained Zero-Shot Forecastingcited by 102
  12. 2024RelBench: A Benchmark for Deep Learning on Relational Databasescited by 90
  13. 2024Better by Default: Strong Pre-Tuned MLPs and Boosted Trees on Tabular Datacited by 87
  14. 2023XTab: Cross-table Pretraining for Tabular Transformerscited by 81
  15. 2024CARTE: Pretraining and Transfer for Tabular Learningcited by 69
  16. 2025Mitra: Mixed Synthetic Priors for Enhancing Tabular Foundation Modelscited by 67
  17. 2025LimiX: Unleashing Structured-Data Modeling Capability for Generalist Intelligencecited by 65
  18. 2023Relational Deep Learning: Graph Representation Learning on Relational Databasescited by 55
  19. 2020Supervised Learning on Relational Databases with Graph Neural Networkscited by 48
  20. 2025Do-PFN: In-Context Learning for Causal Effect Estimationcited by 46
  21. 2025RelGNN: Composite Message Passing for Relational Deep Learningcited by 45
  22. 2025CausalPFN: Amortized Causal Effect Estimation via In-Context Learningcited by 43
  23. 2025Griffin: Towards a Graph-Centric Relational Database Foundation Modelcited by 41
  24. 2021TabularNet: A Neural Network Architecture for Understanding Semantic Structures of Tabular Datacited by 36
  25. 2024PyTorch Frame: A Modular Framework for Multi-Modal Tabular Learningcited by 32
  26. 2025Relational Transformer: Toward Zero-Shot Foundation Models for Relational Datacited by 32
  27. 2025Relational Graph Transformercited by 30
  28. 2025Foundation Models for Causal Inference via Prior-Data Fitted Networkscited by 30
  29. 2025Turning Tabular Foundation Models into Graph Foundation Modelscited by 30
  30. 2023UniTabE: A Universal Pretraining Protocol for Tabular Foundation Model in Data Sciencecited by 28
  31. 2024ClavaDDPM: Multi-relational Data Synthesis with Cluster-guided Diffusion Modelscited by 28
  32. 2024Revisiting Nearest Neighbor for Tabular Data: A Deep Tabular Baseline Two Decades Latercited by 28
  33. 2025TimePFN: Effective Multivariate Time Series Forecasting with Synthetic Datacited by 27
  34. 2021TabGNN: Multiplex Graph Neural Network for Tabular Data Predictioncited by 26
  35. 2025GraphPFN: A Prior-Data Fitted Graph Foundation Modelcited by 20
  36. 2024ContextGNN: Beyond Two-Tower Recommendation Systemscited by 19
  37. 2023GFS: Graph-based Feature Synthesis for Prediction over Relational Databasescited by 16
  38. 2022Row Conditional-TGAN for generating synthetic relational databasescited by 15

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 11.

11 papers.

  1. 2026KumoRFM-2: Scaling Foundation Models for Relational Learningcited by 15
  2. 2026OpenRFM: Dissecting Relational In-Context Learningcited by 4
  3. 2026KGPFN: Unlocking the Potential of Knowledge Graph Foundation Model via In-Context Learningcited by 2
  4. 2026Shaping the Prior: How Synthetic Task Distributions Determine Tabular Foundation Model Qualitycited by 2
  5. 2026RelGT-AC: A Relational Graph Transformer for Autocomplete Tasks in Relational Databasescited by 0
  6. 2026Statistically Indistinguishable, Operationally Distinct: A Formal Barrier for Tabular Foundation Modelscited by 0
  7. 2026InvWeaver: Deductive Feedback for Invariant Synthesis in Interacting-Loop Programscited by 0
  8. 2026PluRel-to-RDB-PFN: Schema-Guided Synthetic Relational Pretrainingcited by 0
  9. 2026Causal Foundation Modelscited by 0
  10. 2026Support-Set Target Leakage in Relational Foundation Models during In-Context Learning: Impact, Detection, and Mitigationcited by 0
  11. 2026Support-Set Target Leakage in Relational Foundation Models during In-Context Learning: Model Dependence and Evaluation Reliabilitycited by 0