# PluRel: Synthetic Data unlocks Scaling Laws for Relational Foundation Models arXiv: https://arxiv.org/abs/2602.04029 · HTML: https://mlsanity.com/paper/2602.04029/ ## Key facts - Cited by 19 papers in the map (data through 14 Oct 2026). - Placed in Privacy-preserving tabular data synthesis, one of 13 directions in Synthetic data and self-improvement for LLMs. - Builds on 31 of its 47 references that are in the map. - First posted on arXiv 3 Feb 2026. ## Placed in - Synthetic data and self-improvement for LLMs / [Privacy-preserving tabular data synthesis](https://mlsanity.com/synth/privacy-preserving-tabular-data-synthesis/) ## Builds on (most cited first, 30 of 31) - [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020) - [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021) - [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022) - [Scaling Vision Transformers](https://mlsanity.com/paper/2106.04560/) (2021) - [Chameleon: Mixed-Modal Early-Fusion Foundation Models](https://mlsanity.com/paper/2405.09818/) (2024) - [TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second](https://mlsanity.com/paper/2207.01848/) (2022) - [A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT](https://mlsanity.com/paper/2302.09419/) (2023) - [Transformers Can Do Bayesian Inference](https://mlsanity.com/paper/2112.10510/) (2021) - [TabICL: A Tabular Foundation Model for In-Context Learning on Large Data](https://mlsanity.com/paper/2502.05564/) (2025) - [Query-Key Normalization for Transformers](https://mlsanity.com/paper/2010.04245/) (2020) - [Small-scale proxies for large-scale Transformer training instabilities](https://mlsanity.com/paper/2309.14322/) (2023) - [TabPFN-2.5: Advancing the State of the Art in Tabular Foundation Models](https://mlsanity.com/paper/2511.08667/) (2025) - [TabDPT: Scaling Tabular Foundation Models on Real Data](https://mlsanity.com/paper/2410.18164/) (2024) - [RelBench: A Benchmark for Deep Learning on Relational Databases](https://mlsanity.com/paper/2407.20060/) (2024) - [REaLTabFormer: Generating Realistic Relational and Tabular Data using Transformers](https://mlsanity.com/paper/2302.02041/) (2023) - [Mitra: Mixed Synthetic Priors for Enhancing Tabular Foundation Models](https://mlsanity.com/paper/2510.21204/) (2025) - [Griffin: Towards a Graph-Centric Relational Database Foundation Model](https://mlsanity.com/paper/2505.05568/) (2025) - [Relational Transformer: Toward Zero-Shot Foundation Models for Relational Data](https://mlsanity.com/paper/2510.06377/) (2025) - [Hierarchical Block Structures and High-resolution Model Selection in Large Networks](https://mlsanity.com/paper/1310.4377/) (2013) - [Turning Tabular Foundation Models into Graph Foundation Models](https://mlsanity.com/paper/2508.20906/) (2025) - [ConTextTab: A Semantics-Aware Tabular In-Context Learner](https://mlsanity.com/paper/2506.10707/) (2025) - [The European Union general data protection regulation: what it is and what it means](https://mlsanity.com/paper/2510.02861/) (2025) - [ClavaDDPM: Multi-relational Data Synthesis with Cluster-guided Diffusion Models](https://mlsanity.com/paper/2405.17724/) (2024) - [Fine-tuned In-Context Learning Transformers are Excellent Tabular Data Classifiers](https://mlsanity.com/paper/2405.13396/) (2024) - [Relational Deep Learning: Challenges, Foundations and Next-Generation Architectures](https://mlsanity.com/paper/2506.16654/) (2025) - [Row Conditional-TGAN for generating synthetic relational databases](https://mlsanity.com/paper/2211.07588/) (2022) - [Joint Relational Database Generation via Graph-Conditional Diffusion Models](https://mlsanity.com/paper/2505.16527/) (2025) - [Post-Hoc Reversal: Are We Selecting Models Prematurely?](https://mlsanity.com/paper/2404.07815/) (2024) - [Scaling Experiments in Self-Supervised Cross-Table Representation Learning](https://mlsanity.com/paper/2309.17339/) (2023) - [Graph-Conditional Flow Matching for Relational Data Generation](https://mlsanity.com/paper/2505.15668/) (2025) ## Built on by (most cited first, 19 of 19) - [KumoRFM-2: Scaling Foundation Models for Relational Learning](https://mlsanity.com/paper/2604.12596/) (2026) - [LimiX-2: A Contextual Mechanism Network Towards General Structured-Data Intelligence](https://mlsanity.com/paper/2609.17488/) (2026) - [Towards Anomaly Detection on Relational Data](https://mlsanity.com/paper/2606.18621/) (2026) - [OpenRFM: Dissecting Relational In-Context Learning](https://mlsanity.com/paper/2606.04320/) (2026) - [Advancing Open and Reproducible Relational Learning: RelArena-α, TabPFN-Rel and RPI](https://mlsanity.com/paper/2608.16319/) (2026) - [RelGT-AC: A Relational Graph Transformer for Autocomplete Tasks in Relational Databases](https://mlsanity.com/paper/2606.03040/) (2026) - [Universal Encoders for Modular Relational Deep Learning](https://mlsanity.com/paper/2606.21434/) (2026) - [InvWeaver: Deductive Feedback for Invariant Synthesis in Interacting-Loop Programs](https://mlsanity.com/paper/2607.05478/) (2026) - [Curriculum Matters: Data-Efficient Relational PFN Pretraining with Synthetic Data](https://mlsanity.com/paper/2607.29120/) (2026) - [PluRel-to-RDB-PFN: Schema-Guided Synthetic Relational Pretraining](https://mlsanity.com/paper/2607.29129/) (2026) - [Incremental Evaluation and Training in Relational Deep Learning](https://mlsanity.com/paper/2608.13023/) (2026) - [Relational Task Generation Language: A Declarative Specification Framework for Relational Deep Learning](https://mlsanity.com/paper/2609.01292/) (2026) - [Support-Set Target Leakage in Relational Foundation Models during In-Context Learning: Impact, Detection, and Mitigation](https://mlsanity.com/paper/2609.36384/) (2026) - [Support-Set Target Leakage in Relational Foundation Models during In-Context Learning: Model Dependence and Evaluation Reliability](https://mlsanity.com/paper/2609.36417/) (2026) - [STEER: Reducing Inference Cost in Relational Foundation Models through Semantically Informed Sampling](https://mlsanity.com/paper/2610.00907/) (2026) - [RelICL: Training-free Relational Learning with Tabular Foundation Models](https://mlsanity.com/paper/2610.01725/) (2026) - [When Does Synthetic Relational Data Teach Models to Use Relations? Tracing Predictive Structure from Pretraining Data to Model Behavior](https://mlsanity.com/paper/2610.03057/) (2026) - [Relational Learning as World Modeling](https://mlsanity.com/paper/2610.16763/) (2026) - [RT-J: Large-Scale Pretraining of Relational Transformers for Context-Efficient Predictions](https://mlsanity.com/paper/2610.17867/) (2026) ## Abstract (arXiv) Relational Foundation Models (RFMs) facilitate data-driven decision-making by learning from complex multi-table databases. However, the diverse relational databases needed to train such models are rarely public due to privacy constraints. While there are methods to generate synthetic tabular data of arbitrary size, incorporating schema structure and primary-foreign key connectivity for multi-table generation remains challenging. Here we introduce PLUREL, a framework to synthesize multi-tabular relational databases from scratch. In a step-by-step fashion, PLUREL models (1) schemas with directed graphs, (2) inter-table primary-foreign key connectivity with bipartite graphs, and, (3) feature distributions in tables via conditional causal mechanisms. The design space across these stages supports the synthesis of a wide range of diverse databases, while being computationally lightweight. Using PLUREL, we observe for the first time that (1) RFM pretraining loss exhibits power-law scaling with the number of synthetic databases and total pretraining tokens, (2) scaling the number of synthetic databases improves generalization to real databases, and (3) synthetic pretraining yields strong base models for continued pretraining on real databases. Overall, our framework and results position synthetic data scaling as a promising paradigm for RFMs.