mlsanityData to 7 Oct 2026

PluRel: Synthetic Data unlocks Scaling Laws for Relational Foundation Models

arXiv 2602.04029 ↗First posted 3 Feb 2026cs.DB cs.AI cs.LGcited by 15

On the map: Synthetic data and self-improvement for LLMs → Privacy-preserving tabular data synthesis

Abstract (arXiv)

Relational Foundation Models (RFMs) facilitate data-driven decision-making by learning from complex multi-table databases. However, the diverse relational databases needed to train such models are rarely public due to privacy constraints. While there are methods to generate synthetic tabular data of arbitrary size, incorporating schema structure and primary-foreign key connectivity for multi-table generation remains challenging. Here we introduce PLUREL, a framework to synthesize multi-tabular relational databases from scratch. In a step-by-step fashion, PLUREL models (1) schemas with directed graphs, (2) inter-table primary-foreign key connectivity with bipartite graphs, and, (3) feature distributions in tables via conditional causal mechanisms. The design space across these stages supports the synthesis of a wide range of diverse databases, while being computationally lightweight. Using PLUREL, we observe for the first time that (1) RFM pretraining loss exhibits power-law scaling with the number of synthetic databases and total pretraining tokens, (2) scaling the number of synthetic databases improves generalization to real databases, and (3) synthetic pretraining yields strong base models for continued pretraining on real databases. Overall, our framework and results position synthetic data scaling as a promising paradigm for RFMs.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Placed in Privacy-preserving tabular data synthesis, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 31 of its 47 references that are in the map. First posted on arXiv 3 Feb 2026.

Builds on

31 of 47 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024ClavaDDPM: Multi-relational Data Synthesis with Cluster-guided Diffusion Models28 May

Other references · 30

  1. 2020Scaling Laws for Neural Language Modelscited by 6,411
  2. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  3. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  4. 2021Scaling Vision Transformerscited by 993
  5. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  6. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 643
  7. 2023A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPTcited by 364
  8. 2021Transformers Can Do Bayesian Inferencecited by 338
  9. 2025TabICL: A Tabular Foundation Model for In-Context Learning on Large Datacited by 267
  10. 2020Query-Key Normalization for Transformerscited by 266
  11. 2023Small-scale proxies for large-scale Transformer training instabilitiescited by 182
  12. 2025TabPFN-2.5: Advancing the State of the Art in Tabular Foundation Modelscited by 115
18 more
  1. 2024TabDPT: Scaling Tabular Foundation Models on Real Datacited by 97
  2. 2023REaLTabFormer: Generating Realistic Relational and Tabular Data using Transformerscited by 93
  3. 2024RelBench: A Benchmark for Deep Learning on Relational Databasescited by 90
  4. 2025Mitra: Mixed Synthetic Priors for Enhancing Tabular Foundation Modelscited by 67
  5. 2025Griffin: Towards a Graph-Centric Relational Database Foundation Modelcited by 41
  6. 2013Hierarchical Block Structures and High-resolution Model Selection in Large Networkscited by 37
  7. 2025Relational Transformer: Toward Zero-Shot Foundation Models for Relational Datacited by 32
  8. 2025ConTextTab: A Semantics-Aware Tabular In-Context Learnercited by 31
  9. 2025Turning Tabular Foundation Models into Graph Foundation Modelscited by 30
  10. 2025The European Union general data protection regulation: what it is and what it meanscited by 30
  11. 2025Relational Deep Learning: Challenges, Foundations and Next-Generation Architecturescited by 22
  12. 2024Fine-tuned In-Context Learning Transformers are Excellent Tabular Data Classifierscited by 21
  13. 2022Row Conditional-TGAN for generating synthetic relational databasescited by 15
  14. 2025Joint Relational Database Generation via Graph-Conditional Diffusion Modelscited by 8
  15. 2024Post-Hoc Reversal: Are We Selecting Models Prematurely?cited by 7
  16. 2023Scaling Experiments in Self-Supervised Cross-Table Representation Learningcited by 4
  17. 2025Graph-Conditional Flow Matching for Relational Data Generationcited by 3
  18. 2025Generating Synthetic Relational Tabular Data via Structural Causal Modelscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 15.

15 papers.

Sort:

  1. 2026KumoRFM-2: Scaling Foundation Models for Relational Learningcited by 15
  2. 2026OpenRFM: Dissecting Relational In-Context Learningcited by 4
  3. 2026Towards Anomaly Detection on Relational Datacited by 4
  4. 2026LimiX-2: A Contextual Mechanism Network Towards General Structured-Data Intelligencecited by 3
  5. 2026Advancing Open and Reproducible Relational Learning: RelArena-α, TabPFN-Rel and RPIcited by 2
  6. 2026RelGT-AC: A Relational Graph Transformer for Autocomplete Tasks in Relational Databasescited by 0
  7. 2026Universal Encoders for Modular Relational Deep Learningcited by 0
  8. 2026InvWeaver: Deductive Feedback for Invariant Synthesis in Interacting-Loop Programscited by 0
  9. 2026PluRel-to-RDB-PFN: Schema-Guided Synthetic Relational Pretrainingcited by 0
  10. 2026Incremental Evaluation and Training in Relational Deep Learningcited by 0
  11. 2026Relational Task Generation Language: A Declarative Specification Framework for Relational Deep Learningcited by 0
  12. 2026Support-Set Target Leakage in Relational Foundation Models during In-Context Learning: Impact, Detection, and Mitigationcited by 0
3 more
  1. 2026Support-Set Target Leakage in Relational Foundation Models during In-Context Learning: Model Dependence and Evaluation Reliabilitycited by 0
  2. 2026STEER: Reducing Inference Cost in Relational Foundation Models through Semantically Informed Samplingcited by 0
  3. 2026RelICL: Training-free Relational Learning with Tabular Foundation Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026When Does Synthetic Relational Data Teach Models to Use Relations? Tracing Predictive Structure from Pretraining Data to Model Behavior
  2. 2025RelDiff: Relational Data Generative Modeling with Graph-Based Diffusion Models
  3. 2024Enhancing Table Representations with LLM-powered Synthetic Data Generation
  4. 2025Risk In Context: Benchmarking Privacy Leakage of Foundation Models in Synthetic Tabular Data Generation
  5. 2025LLM-TabLogic: Preserving Inter-Column Logical Relationships in Synthetic Tabular Data via Prompt-Guided Latent Diffusion
  6. 2024Differentially Private Tabular Data Synthesis using Large Language Models
  7. 2024Towards Privacy-Preserving Relational Data Synthesis via Probabilistic Relational Models
  8. 2026Creation begins with understanding: LLMs as strategy designers for privacy-preserving tabular data synthesis