# KumoRFM-2: Scaling Foundation Models for Relational Learning

arXiv: https://arxiv.org/abs/2604.12596 · HTML: https://mlsanity.com/paper/2604.12596/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 45 references that are in the map.
- First posted on arXiv 14 Apr 2026.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second](https://mlsanity.com/paper/2207.01848/) (2022)
- [Heterogeneous Graph Transformer](https://mlsanity.com/paper/2003.01332/) (2020)
- [Principal Neighbourhood Aggregation for Graph Nets](https://mlsanity.com/paper/2004.05718/) (2020)
- [AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data](https://mlsanity.com/paper/2003.06505/) (2020)
- [TabICL: A Tabular Foundation Model for In-Context Learning on Large Data](https://mlsanity.com/paper/2502.05564/) (2025)
- [TabICLv2: A better, faster, scalable, and open tabular foundation model](https://mlsanity.com/paper/2602.11139/) (2026)
- [RelBench: A Benchmark for Deep Learning on Relational Databases](https://mlsanity.com/paper/2407.20060/) (2024)
- [LimiX: Unleashing Structured-Data Modeling Capability for Generalist Intelligence](https://mlsanity.com/paper/2509.03505/) (2025)
- [Mitra: Mixed Synthetic Priors for Enhancing Tabular Foundation Models](https://mlsanity.com/paper/2510.21204/) (2025)
- [CARTE: Pretraining and Transfer for Tabular Learning](https://mlsanity.com/paper/2402.16785/) (2024)
- [Relational Deep Learning: Graph Representation Learning on Relational Databases](https://mlsanity.com/paper/2312.04615/) (2023)
- [RelGNN: Composite Message Passing for Relational Deep Learning](https://mlsanity.com/paper/2502.06784/) (2025)
- [Griffin: Towards a Graph-Centric Relational Database Foundation Model](https://mlsanity.com/paper/2505.05568/) (2025)
- [Real-TabPFN: Improving Tabular Foundation Models via Continued Pre-training With Real-World Data](https://mlsanity.com/paper/2507.03971/) (2025)
- [Relational Transformer: Toward Zero-Shot Foundation Models for Relational Data](https://mlsanity.com/paper/2510.06377/) (2025)
- [Turning Tabular Foundation Models into Graph Foundation Models](https://mlsanity.com/paper/2508.20906/) (2025)
- [Relational Graph Transformer](https://mlsanity.com/paper/2505.10960/) (2025)
- [ConTextTab: A Semantics-Aware Tabular In-Context Learner](https://mlsanity.com/paper/2506.10707/) (2025)
- [Fine-tuned In-Context Learning Transformers are Excellent Tabular Data Classifiers](https://mlsanity.com/paper/2405.13396/) (2024)
- [RelBench v2: A Large-Scale Benchmark and Repository for Relational Data](https://mlsanity.com/paper/2602.12606/) (2026)
- [Orion-MSP: Multi-Scale Sparse Attention for Tabular In-Context Learning](https://mlsanity.com/paper/2511.02818/) (2025)
- [PluRel: Synthetic Data unlocks Scaling Laws for Relational Foundation Models](https://mlsanity.com/paper/2602.04029/) (2026)
- [Bringing Graphs to the Table: Zero-shot Node Classification via Tabular Foundation Models](https://mlsanity.com/paper/2509.07143/) (2025)
- [Tackling prediction tasks in relational databases with LLMs](https://mlsanity.com/paper/2411.11829/) (2024)
- [Relational In-Context Learning via Synthetic Pre-training with Structural Prior](https://mlsanity.com/paper/2603.03805/) (2026)

## Built on by (most cited first, 19 of 19)

- [TabPFN-3: Technical Report](https://mlsanity.com/paper/2605.13986/) (2026)
- [OpenRFM: Dissecting Relational In-Context Learning](https://mlsanity.com/paper/2606.04320/) (2026)
- [Advancing Open and Reproducible Relational Learning: RelArena-α, TabPFN-Rel and RPI](https://mlsanity.com/paper/2608.16319/) (2026)
- [RelAgent: LLM Agents as Data Scientists for Relational Learning](https://mlsanity.com/paper/2605.07840/) (2026)
- [Incorporating Deep Learning Design in Database Queries](https://mlsanity.com/paper/2605.24207/) (2026)
- [RelGT-AC: A Relational Graph Transformer for Autocomplete Tasks in Relational Databases](https://mlsanity.com/paper/2606.03040/) (2026)
- [FlexTab: A Flexible Encoder-Decoder Architecture for In-Context Learning Across Diverse Tabular Tasks](https://mlsanity.com/paper/2606.30336/) (2026)
- [InvWeaver: Deductive Feedback for Invariant Synthesis in Interacting-Loop Programs](https://mlsanity.com/paper/2607.05478/) (2026)
- [Incremental Evaluation and Training in Relational Deep Learning](https://mlsanity.com/paper/2608.13023/) (2026)
- [Relational Task Generation Language: A Declarative Specification Framework for Relational Deep Learning](https://mlsanity.com/paper/2609.01292/) (2026)
- [Reification as a Transferable Vocabulary: Zero-Shot Link Prediction with Vanilla GNNs](https://mlsanity.com/paper/2609.11347/) (2026)
- [The Universal Classifier for Graph Learning](https://mlsanity.com/paper/2609.36302/) (2026)
- [Support-Set Target Leakage in Relational Foundation Models during In-Context Learning: Impact, Detection, and Mitigation](https://mlsanity.com/paper/2609.36384/) (2026)
- [Support-Set Target Leakage in Relational Foundation Models during In-Context Learning: Model Dependence and Evaluation Reliability](https://mlsanity.com/paper/2609.36417/) (2026)
- [STEER: Reducing Inference Cost in Relational Foundation Models through Semantically Informed Sampling](https://mlsanity.com/paper/2610.00907/) (2026)
- [RelICL: Training-free Relational Learning with Tabular Foundation Models](https://mlsanity.com/paper/2610.01725/) (2026)
- [Relational Learning as World Modeling](https://mlsanity.com/paper/2610.16763/) (2026)
- [RT-J: Large-Scale Pretraining of Relational Transformers for Context-Efficient Predictions](https://mlsanity.com/paper/2610.17867/) (2026)
- [Revisiting Simple Baselines for Relational Deep Learning](https://mlsanity.com/paper/2610.20534/) (2026)

## Abstract (arXiv)

We introduce KumoRFM-2, the next iteration of a pre-trained foundation model for relational data. KumoRFM-2 supports in-context learning as well as fine-tuning and is applicable to a wide range of predictive tasks. In contrast to tabular foundation models, KumoRFM-2 natively operates on relational data, processing one or more connected tables simultaneously without manual table flattening or target variable generation, all while preserving temporal consistency. KumoRFM-2 leverages a large corpus of synthetic and real-world data to pre-train across four axes: the row and column dimensions at the individual table level, and the foreign key and cross-sample dimensions at the database level. In contrast to its predecessor, KumoRFM-2 injects task information as early as possible, enabling sharper selection of task-relevant columns and improved robustness to noisy data. Through extensive experiments on 41 challenging benchmarks and analysis around expressivity and sensitivity, we demonstrate that KumoRFM-2 outperforms supervised and foundational approaches by up to 8%, while maintaining strong performance under extreme settings of cold start and noisy data. To our knowledge, this is the first time a few-shot foundation model has been shown to surpass supervised approaches on common benchmark tasks, with performance further improving upon fine-tuning. Finally, while KumoRFM-1 was limited to small-scale in-memory datasets, KumoRFM-2 scales to billion-scale relational datasets.